Enhance Your Jobs Feeds with Our New Salary API
Blog
6 December 2012
Synchronize via incremental job scraping
Keep your job listings up-to-date with source Employer websites by utilizing Synchronization option of Job Wrapping. Sync option will make sure only new jobs are added to your system and expire the vacancies removed from source websites.
Incremental scraping
Job wrapping synchronization feature provides incremental downloads to make sure you don’t overload source websites with excessive requests, but only download new jobs added to system.
Job Sync requirements:
Job Sync feature requires HTTP/XML posting interface on a target job board.The interface is to provide “Add” and “Remove” commands. It is also to be able to return job ID to spider upon successful posting.
Sync process:
1. Job wrapping runs search on Employer career center and assigns unique ID to each job / specific URL scraped.
2. Spider posts the job via HTTP/XML: posting interface of a Job Board returns unique job ID of each specific job posting.
3. Job wrapping service saves job ID from job board and links with Job ID of a Spider.
4. Next job scraping sessions are done by spider. If the job is gone from job search results: job spider sends Remove ID command to job board XML interface.
5. Job board removes/expires the job.
Featured posts
JobsIndex Data vs. BLS Data: What's the Difference?
Every month, the US Bureau of Labor Statistics (BLS) publishes...
26 September 2024
Finally: Real-Time Labor Market Data
In November 2022, Amazon stock hit a 12-month low ($84.18),...
22 May 2024
SAG-AFTRA, UAW, Amazon, and More: 3 Hiring Takeaways from Today's High-Profile Strikes
Labor issues have upended how employers and workers alike currently...
11 January 2024