Just to confirm: Indeed.com prohibits spidering of its content and they will block anyone trying to scrape it.
Normally, our clients ask us to spider jobs from direct employer websites and ATSes.
In some cases we can spider commercial job boards: if there is a formal agreement between our client and the job board to allow spidering.
Download our Job Scraping White Paper
If you are new to the jobs industry or just starting your job board, you need to ask these 8 questions about job scraping.
Download our 2022 Job Market Pulse Snapshot
This report highlights a variety of insights that can be drawn from our database of more than 9 million vacancies worldwide from over 100,000 different companies, updated daily