Crawler Information

What We Do

Rollie Jobs partners with chambers of commerce and economic development organizations to help promote local job opportunities. We discover public job postings and surface them to help regional business leaders understand their local employment landscape.

Crawling Practice

  • Frequency: Weekly crawls maximum

  • Scope: Public job postings only

  • Traffic Direction: All job seekers directed to original job posting

  • Purpose: Local economic development and workforce intelligence

  • Respect: We honor robots.txt and rate limit all requests

Crawling Policy

  • Respect robots.txt directives

  • Rate limited requests (max 1 request per 3 seconds)

  • No paywall circumvention

  • Public data only

  • Weekly frequency maximum per domain

Data Usage

  • Job postings aggregated for local economic intelligence

  • No job applications processed on our platform

  • All applicants directed to original job posting

  • Data used to help partners understand local workforce and economic trends

Data Removal

Removal of data can be requested by contacting info@rolliejobs.com.

Rollie Jobs is committed to ethical web crawling practices and building mutually beneficial relationships with job boards and employers.