The Value of Job Scraping Today: A Q&A with Aspen Tech Labs Founder Michael Woodrow

Our CEO and founder, Michael Woodrow, recently had the pleasure of speaking on the podcast Job Board Secrets, which sources insights from various professionals in the recruiting space on how to build more effective job boards. 

In this conversation, Michael elaborates on the hurdles job boards face and how Aspen Tech Lab’s services – specifically our JobIndex tool – help clients clear them. 

Here, we’ll outline and expand on the insights Michael presented. 

For those who don’t know, what does Aspen Tech Labs do?

We provide web data management for various industries (autos, classified, property, and jobs). We also offer access to some of the considerable data we collect, monitor, and standardize. 

The product most relevant to Job Board Secrets listeners is our JobsIndex tool, which includes more than eight million unique, up-to-date organic job listings. 

But before diving deeper, it’s important to understand why this work is crucial in today’s job market. In the US, there were 25.6 million resignations in the second half of 2021 alone. And as I mentioned, there are about eight million open jobs right now.

Can you explain the JobsIndex a bit more?

Certainly. JobsIndex provides organic jobs content, straight from various ATSes, which means applicants who click through get the employer’s site, not another job board. This saves the candidate time and improves the UX of the job board.

And because JobsIndex sources listings globally, from more than 50,000 corporate career sites, we can support international recruiting efforts while providing a wide range of positions.

Do job boards get paid when applicants click on these listings?

No. 

These listings are about improving the user experience of a job board. 

Think about it like this: an engineer from Chicago visits your site and only sees three engineering jobs based in Chicago. They will likely leave. But with the organic listings, you have a lot more to offer, which helps prevent people from clicking away.  This provides for an enhanced job seeker experience, increasing the likelihood that she will return later to the job board.

What’s the advantage to providing organic listings, beyond user experience?

This past month, there were millions of searches for “jobs near me.” This demonstrates two things:

1. Plenty of people are looking for jobs.
2. Candidates want to know who is hiring near them.

Job boards interested in tapping some of that search traffic can request a JobsIndex feed that includes geographic (eg jobs in Chicago)  jobs. When those people find these jobs and click through to their site, job boards can share not only these organic listings, but also their relevant sponsored listings.

So customers can request listings from specific regions or areas of specialization?

Yes. Customers frequently come to us and ask for something like 10,000 organic jobs, refining their request by location or industry. 

For example, a customer may request 5,000 jobs, but limit the scope to employers based in the Pacific Northwest that are hiring for IT roles. We then apply that filter to our JobsIndex and deliver 5,000 relevant listings.

We also offer custom job scraping, which can provide clean jobs content from specific companies,, with additional levels of monitoring and data management.

Do customers ever use your jobs data to gain insight into market trends?

We’ve noticed that’s a growing use case, yes, particularly for recruitment advertising agencies, business strategy and sales prospecting. 

We base our on-demand analytics on freshly scraped jobs so customers can see how the employment market is changing (which companies are posting, which jobs are popular, etc.). These analytics include easy-to-read visualizations, demonstrating what’s happening in the job market.

Any trends job board leaders should keep an eye on?

Yes, I have two.

The first is Google for Jobs. Point blank: you need to optimize your jobs content for the Google for Jobs algorithm. Doing so typically gets you more traffic, which can boost your revenue.

Second, we need to discuss the Cost per Application (CPA) job board revenue model. We know rates are growing as companies invest further in recruitment. Now, it’s not about receiving $5 per application – it’s about receiving $50. 

Job boards can capitalize on this more lucrative structure. If you receive $50 per application and one person applied but 10 people clicked, each click is now worth $5, instead of maybe $.20-$1.00 on a CPC model.

What should job boards who are interested in your services know about reaching out?

We offer month-long free trials. That’s how confident we are in the value we provide. Just let us know what jobs you’re interested in and the right delivery method. We’ll get fresh content to you in a matter of days.

If you’re looking to cut costs while building your job board with millions of updated listings, contact us here for a demo.