System IconControl Room

Real-time scrape monitoring and control.

Tags:
Select All Scrapers
Sort by:
Status:
to
  • Loading scrapers...

Sponsor Register Sync

gov.uk/ukvi-sponsorsNot Yet Updated
Schedule: 11:30am, 12:30pm, 1:30pm, 7pm Mon-Sat
Last Checked: Never
Last Synced: Never
Gov.uk Data Date: Unknown
Sync Integrity

Loading health data...

No sponsor sync runs recorded yet.

Visa v2 Data Source Refresh

gov.uk · 0 sourcesNo data
Skilled Worker + Health & Care eligibility inputs

Overview

(last cron heartbeat per source)
0 of 0 checked today0 with new data

Chk = system check time (every cron tick). Sync = last time NEW data was ingested (quarterly/monthly cadence depending on the source). Green dot = checked today; red dot = not yet checked today. Gold star = new data synced within the last 5 days. Click a row to jump to the card.

Immigration System Statistics Data

(0/2)
No Immigration system statistics sources returned by the API yet.

Skilled Worker Visa

(0/5)
No Skilled Worker sources returned by the API yet.

Health & Care Worker Visa

(0/3)
No Health & Care sources returned by the API yet.

Policy Watch

(0/1)— early-warning signals for rule changes
No policy-watch sources returned by the API yet (migration 048 may not have run).

How it Works

Adding New Scrapers

You cannot create a logic scraper directly from this interface. To add a new scraper:

  1. Create a new Python file in src/scrapers/plugins/
  2. Extend BaseScraper and add the @register decorator.
  3. Restart the server. The script init_db.py will auto-detect it and it will permanently appear here.

Schedules & Job Volumes

The system uses Cron definitions (e.g., 0 */4 * * *) to automatically trigger scrapers every few hours.

  • You can manually click "Run Now" to trigger an ad-hoc run at any time regardless of the schedule.
  • By default, scrapers are programmed to fetch all available listings they find up to a hardcoded page limit inside their Python plugin to prevent rate-limiting.
  • If a target site blocks your IP, the Status will say FAILED. Check logs for the Exception trace.