Blog

Synchronize via incremental job scraping

Keep your job listings up-to-date with source Employer websites by utilizing Synchronization option of Job Wrapping. Sync option will make sure only new jobs are added to your system and expire the vacancies removed from source websites.

Incremental scraping

Job wrapping synchronization feature provides incremental downloads to make sure you don’t overload source websites with excessive requests, but only download new jobs added to system.

Job Sync requirements:

Job Sync feature requires HTTP/XML posting interface on a target job board.The interface is to provide “Add” and “Remove” commands. It is also to be able to return job ID to spider upon successful posting.

Sync process:

1. Job wrapping runs search on Employer career center and assigns unique ID to each job / specific URL scraped.
2. Spider posts the job via HTTP/XML: posting interface of a Job Board returns unique job ID of each specific job posting.
3. Job wrapping service saves job ID from job board and links with Job ID of a Spider.
4. Next job scraping sessions are done by spider. If the job is gone from job search results: job spider sends Remove ID command to job board XML interface.
5. Job board removes/expires the job.

Follow us