SpiderMount job wrapping service includes standard job taxonomy feature built-in. Job taxonomy automatically defines job category/industry based on job titles and keywords.
Generic job taxonomy included into the job wrapping standard service maps jobs with the predefined list of high-level industry categories. Generic categories are mapped with job board’s sector/industry list.
SpiderMount recognizes uncategorized jobs from the scraped list and sets relevant category from generic industry list with a further update in posting feed. The generic option does not require custom job title mappings provided by a client but uses generic mappings for 40 top level industry categories matched to client listings.
Alternatively, a custom taxonomy service can be configured if the genetic taxonomy is not sufficient for correct category mapping.
Generic high-level industry categories listing:
For more information, please contact SpiderMount team.
Learn more about job taxonomy service:
We noticed you mentioned scraping Indeed.com
Just to confirm: Indeed.com prohibits spidering of its content and they will block anyone trying to scrape it.
Normally, our clients ask us to spider jobs from direct employer websites and ATSes.
In some cases we can spider commercial job boards: if there is a formal agreement between our client and the job board to allow spidering.