Using Mapping tool you can see/update mapping of incoming data (source) and your required IDs or values (target data). Once there are new values that require mapping you will see the following icon near a scrape, that has new non-mapped values:
To map new values click the mapping icon near a scrape or on the left-hand navigation panel to see the list of dictionaries and details about the number of new and existing values:
And after opening dictionary mapping page there are two sections : the yellow one is for the new non-mapped values and the green one is for the existing mappings:
Click + button to map new values and search by name e.g. “Work from home” or by ID e.g. 5430.
Note that you can map each source value to one or more target values.
Also, you can search by keywords across all new values, tick similar values, and map them in bulk:
Once mapping of new values is done it’s important to re-process related scrapes.
“Require mapping” icon will turn to “Restart required” icon and you can restart selected scrapes from the scraping dashboard with the button
Or you can restart all scrapes that have new mapping values and require update from the mapping list page:
We noticed you mentioned scraping Indeed.com
Just to confirm: Indeed.com prohibits spidering of its content and they will block anyone trying to scrape it.
Normally, our clients ask us to spider jobs from direct employer websites and ATSes.
In some cases we can spider commercial job boards: if there is a formal agreement between our client and the job board to allow spidering.
Download the first Job Market Pulse Quarterly U.S. Jobs Report for 2023. It focuses on bringing you quarterly job market trends and insights from our database of more than 120,000 companies and 6M U.S. job postings.