We noticed you mentioned scraping Indeed.com
Just to confirm: Indeed.com prohibits spidering of its content and they will block anyone trying to scrape it.
Normally, our clients ask us to spider jobs from direct employer websites and ATSes.
In some cases we can spider commercial job boards: if there is a formal agreement between our client and the job board to allow spidering.
If you are new to the jobs industry or just starting your job board, you need to ask these 8 questions about job scraping.
This report highlights a variety of insights that can be drawn from our database of more than 9 million vacancies worldwide from over 100,000 different companies, updated daily
