One question we frequently hear from job boards is: “Which delivery method is better, XML or API?” Before answering that question, it’s important to know there isn’t a delivery method that is “the best.”
The right delivery method for your job board depends on factors like the number of listings you provide, whether you have a development team, and how much of the scraping process you want to own or outsource.
That’s why we’ve created this table with sample use cases to help you identify which delivery method is right for you. While this table only compares XML vs. API delivery, there are other formats to consider (CSV, JSON, etc.). If you’d like any help discussing the best delivery method for you, reach out.
|Use Case||Delivery Format: XML vs. API||Reason|
|You need real-time updates to your jobs data.||API||- Enables a job scraping team to continuously track changes on source sites and initiate updates to job postings once changes appear.|
- Lets your team target job listings that need to be updated (new descriptions, invalid links, etc.) or removed (filled positions).
|You have a limited budget and timeline to set up a delivery method.||XML||- Reduces the costs of hiring staff to manage the implementation and ongoing jobs delivery.|
- Takes a fraction of the time to set up.
|Your job board offers 100,000 jobs or fewer.||XML||- Allows job board admins to independently initiate updates to jobs data.|
- Processes bulk updates(aka changes to jobs data) efficiently.
|Your job board offers a million jobs or more.||API||- Enables targeted updates to specific jobs.|
- Saves you the time it would take to bulk update the millions of jobs – as you would do with XML.
- Tracks all job updates continuously, which ensures your listings remain up to date and clean.
|You have between 100,000 and 1 million jobs||Depends||In this gray area, we’d look to your other needs and the other features of your job board.|
|You already have an API interface for job posting.||API||- Leverages an existing interface for features like real-time posting and updates.|
- Saves you the time of establishing a new delivery method.
|You have a small or nonexistent developer team.||XML||- Simplifies jobs data management (no highly technical configurations).|
- Enables one person to fully control the import process and scheduling of job updates / postings.
|You want automated job management.||API||- Rotates in new jobs from your database to replace deleted postings.|
- Displays the most relevant jobs using various category IDs (sponsored vs. organic, industry, title, etc.).
What should you do if more than one of these use cases applies to you? Get in touch. We can help you determine which delivery format (XML vs. API) is best suited to your particular needs.
Identifying the right delivery method for your job board takes time, but it’s only one part of the job scraping process. You can have the perfect delivery method but if you don’t conduct daily monitoring for your scrapes, they can break and lose effectiveness.
If you’re spending too much time or money on web scraping and you’re not seeing the results you want, request a demo. We build and monitor scrapes more accurately and more cost effectively than what you can do in-house. And we deliver the jobs data in the format that best suits your needs.
We noticed you mentioned scraping Indeed.com
Just to confirm: Indeed.com prohibits spidering of its content and they will block anyone trying to scrape it.
Normally, our clients ask us to spider jobs from direct employer websites and ATSes.
In some cases we can spider commercial job boards: if there is a formal agreement between our client and the job board to allow spidering.