Most of the modern job boards are benefiting from job wrapping or spidering services. However, they still encounter an issue with extracting correct industry category for the vacancies posted by employers. SpiderMount introduces new service to resolve the problem for automated job scraping – job taxonomy.
Job Taxonomy is automated industry/category recognition with the mapping based on available job data, i.e. job title. Online service analyses jobs’ content and defines relevant industry category data.
There are two options available to integrate service into your current solution or service:
Job boards, ATSes, online recruitment services can send jobs XML data to JobTaxonomy API and obtain Industry category IDs mapped to jobs.
Job spidering service in combination with JobTaxonomy scrapes jobs from employers’ websites lacking Industry Category listings and automatically defines relevant job industry sector/category with further update in the scraped data.
Learn more about job scraping features.
We noticed you mentioned scraping Indeed.com
Just to confirm: Indeed.com prohibits spidering of its content and they will block anyone trying to scrape it.
Normally, our clients ask us to spider jobs from direct employer websites and ATSes.
In some cases we can spider commercial job boards: if there is a formal agreement between our client and the job board to allow spidering.