Jobs Indexing and Extraction service collects millions of fresh jobs daily from thousands of original (corporate) sources globally.
Aspen Technology Labs Inc. builds and maintains one of the largest sources of job market data, including millions of jobs, updated daily, from top firms in such industries as financial services, healthcare, transportation and others.
Use of this data includes backfill for niche job boards/aggregators, competitive intelligence, analytics, public relations and candidate sourcing. It is also useful to identify insights to the worldwide (or a niche) labor market. Customers use this fresh, organic jobs content for their job boards and job alerts. Savvy customers identify gaps in their jobs content (for example common search results that return too few jobs), and fill the gaps with JobsIndEx jobs.
We use our stack of technologies to retrieve specific job pages from various company sources and ATSes and extract the information related to jobs.
Our system, under the direction of our engineers, sets up rules of taxonomy mappings and classification to normalize the jobs data.
We apply our enhancement engine to additionally categorize, geocode, validate and format the jobs data to fit your specific needs. For example, we map your feed to your categories, or, you need us to change application links to add a tracking code.
The data over the Internet is continuously changing, whereas the data in the JobsIndEx’s database must be consistent. To achieve this, JobsIndEx is being automatically monitored using our tech tools, and our QA team of engineers receive alerts, so they can investigate and resolve any issues.
We noticed you mentioned scraping Indeed.com
Just to confirm: Indeed.com prohibits spidering of its content and they will block anyone trying to scrape it.
Normally, our clients ask us to spider jobs from direct employer websites and ATSes.
In some cases we can spider commercial job boards: if there is a formal agreement between our client and the job board to allow spidering.