# Penelope REST API Penelope exposes a persistent, non-headless Firefox (Playwright/Patchright) through a REST API. The server keeps a pool of browser pages ("tabs"); every operation targets one page, identified by a `page_id`. - Base URL: `http://:5000` - All API routes are prefixed with `/api/v1` - Interactive OpenAPI docs: `/docs` (Swagger) and `/redoc` - Control panel UI: `/` ## Layers | Layer | Prefix | Purpose | |---|---|---| | System | `/api/v1/system` | Browser/server lifecycle, health, pool status | | Ops | `/api/v1/ops` | Page management: create, destroy, navigate, extract, screenshot | | Primitive | `/api/v1/primitive` | Atomic actions: click, scroll, type, paste, keys, zoom | | Plugins | `/api/v1` | High-level workflows built on top of the primitives | --- ## Authentication Every endpoint that touches the browser requires a bearer token matching the server's `PENELOPE_KEY` environment variable: ``` Authorization: Bearer ``` Unauthenticated endpoints (no header needed): - `GET /api/v1/system/ping` - `GET /api/v1/system/status` - `GET /api/v1/system/status/pages` - `GET /api/v1/plugins` Failure modes: - `401` — missing or wrong token (`{"detail": "Invalid API key"}`) - `500` — server started without `PENELOPE_KEY` set (`{"detail": "Server not properly configured"}`) --- ## Conventions ### Page IDs A `page_id` is an arbitrary string naming a browser tab (`main`, `scraper1`, ...). - Pages are **not** created implicitly. Create one with `POST /api/v1/ops/create-page` before using it; operating on an unknown id fails with `Page not found`. - `POST /api/v1/ops/create-page` with no body (or `page_id: null`) auto-generates the id and returns it. - The pool is capped at `MAX_PAGES` (default 5, set in `server.py`). Page states reported by the pool: `idle`, `busy`, `loading`, `error`, `crashed`, `stuck`. ### Requests All request bodies are JSON (`Content-Type: application/json`). Endpoints that take no parameters accept an empty body. ### Responses Successful responses share a base shape and add endpoint-specific fields: ```json { "success": true, "page_id": "scraper", "...": "endpoint-specific fields" } ``` Errors raised by the application layer use the `ErrorResponse` envelope: ```json { "success": false, "error": "Page scraper not found", "page_id": "scraper" } ``` Status codes: `400` bad request / plugin validation failure, `401` unauthorized, `408` navigation timeout, `500` browser navigator or page pool not initialized. Note: some browser-level failures are reported *inside a `200` response* with `"success": false` and an `error` field, rather than as an HTTP error. Always check `success`, not only the status code. --- ## System ### `GET /api/v1/system/ping` Health check. Sleeps ~2s, then returns a timestamp. No auth. ```json { "pong": 1739381923123, "status": "alive" } ``` ### `GET /api/v1/system/status` Browser navigator status. No auth. ```json { "initialized": true, "browser_context_active": true, "page_pool_active": true, "max_pages": 5, "slowmo": 200, "page_count": 2, "available_pages": 1 } ``` `page_count` and `available_pages` are `null` when the pool is not up. When the navigator is stopped, `initialized` is `false` and the numeric fields are `0`. ### `GET /api/v1/system/status/pages` Detailed pool status. No auth. `500` if the navigator or pool is not initialized. ```json { "total_pages": 2, "max_pages": 5, "status_by_state": { "idle": 1, "busy": 1 }, "pages": [ { "page_id": "scraper", "state": "idle", "current_operation": null, "operation_duration": null, "responsive": true, "loading": false, "stuck": false, "error_count": 0, "last_activity": 1739381923.12, "url": "https://example.com", "title": "Example Domain" } ] } ``` ### `GET /api/v1/system/start` Auth required. Starts the browser navigator; if one is already running it is closed first (restart). All existing pages are lost. ```json { "success": true, "page_id": null, "message": "Browser navigator started successfully" } ``` ### `GET /api/v1/system/stop` Auth required. Closes the browser and all pages. The HTTP server keeps running. ### `GET /api/v1/system/kill` Auth required. Triggers graceful shutdown of the whole server process. --- ## Ops — page management All Ops endpoints require auth. ### `POST /api/v1/ops/create-page` Body (optional): ```json { "page_id": "scraper" } ``` Response: ```json { "success": true, "page_id": "scraper", "message": "Page 'scraper' created successfully", "url": "about:blank", "state": "idle", "total_pages": 1 } ``` `400` if the page cannot be created (e.g. duplicate id, pool full). ### `POST /api/v1/ops/{page_id}/destroy-page` ### `DELETE /api/v1/ops/{page_id}/destroy-page` Both verbs do the same thing: close and remove the page. ```json { "success": true, "page_id": "scraper", "message": "Page 'scraper' destroyed", "total_pages": 0 } ``` ### `POST /api/v1/ops/{page_id}/navigate` Navigates and waits for `domcontentloaded`. The handler sleeps ~2s before starting and aborts with `408` after 60s. ```json { "url": "https://example.com" } ``` ```json { "success": true, "page_id": "scraper", "url": "https://example.com", "status_code": 200, "final_url": "https://example.com/" } ``` ### `POST /api/v1/ops/{page_id}/extract-content` No body. Returns the full HTML of the current document. ```json { "success": true, "page_id": "scraper", "url": "https://example.com/", "title": "Example Domain", "content": "…", "content_length": 1256 } ``` ### `GET /api/v1/ops/{page_id}/screenshot` Returns two base64-encoded PNGs: the viewport and the full page. ```json { "success": true, "page_id": "scraper", "screenshot": "iVBORw0KG…", "full_screenshot": "iVBORw0KG…", "url": "https://example.com/" } ``` ### `POST /api/v1/ops/{page_id}/search-object` Vision-based object lookup on the current page (screenshot → Gemini). Same engine as the `detect-bounding-box` plugin. ```json { "description": "the blue submit button" } ``` ```json { "success": true, "page_id": "scraper", "description": "the blue submit button", "bounding_box": { "…": "model-dependent" }, "center_x": 412.0, "center_y": 233.0, "tokens_used": 1834, "costs_usd": 0.0004, "duration_seconds": 2.71 } ``` Requires a Gemini API key in the server environment (`GEMINI_API_KEY_PAGA` / `GEMINI_API_KEY_GRATIS`). Pair it with `click-position` to click what was found. --- ## Primitive — atomic actions All Primitive endpoints require auth and return at least `success`, `page_id` and `url` (the page URL after the action). > Keyboard primitives (`type-text`, `paste-text`, `press-enter`, `press-tab`) act on the > **currently focused element**. Focus something first — usually with `click-element`. ### `POST /api/v1/primitive/{page_id}/click-element` ```json { "selector": "input[name=\"q\"]" } ``` Response adds `selector`. ### `POST /api/v1/primitive/{page_id}/click-position` ```json { "x": 100, "y": 200 } ``` Response adds `position`. ### `POST /api/v1/primitive/{page_id}/check-visible` Checks whether an element exists, is rendered, and is fully inside the viewport (not clipped by scrolling or container overflow). ```json { "selector": "header nav" } ``` Response adds `selector`, `found`, `visible` (at least partially on screen), `fully_visible`, `bounding_box` (full layout rect), `intersection` (the actually visible portion — element rect clipped to the viewport and any overflow-clipping ancestors), `viewport`. ### `POST /api/v1/primitive/{page_id}/scroll-page` ```json { "distance": 1000 } ``` `distance` is optional (default `1000`), in pixels; **negative scrolls up**. There is no `direction` field — direction is only the sign of `distance` (an unknown field like `"amount"` or `"direction"` is silently ignored and the default 1000 is used). `scrolled_distance` in the response is the **gross** distance actually emitted during the main scroll, not the net viewport shift: - the request is inflated by a random overshoot factor ×1.1–1.2 - the overshoot is then partially recovered with a small back-scroll of ×0.1–0.2 of the inflated distance (net ≈ the requested distance, ±10%) - the scroll is emitted as a burst of wheel notches of ~80–120 px each with human-like pacing (slow at start/end, fast in the middle); the page can keep settling after the call returns Example: `distance: 500` → overshoot ~570, recovery ~−80, `scrolled_distance` reports ~570 while the page actually moved ~500. Because of the ±10% tolerance, never use scroll for precise positioning: scroll, then verify with `check-visible` and correct with a small signed `distance` if the target is not `fully_visible` yet. Response adds `scrolled_distance`. ### `POST /api/v1/primitive/{page_id}/type-text` Types character by character with human-like timing (and occasional corrected typos). ```json { "text": "hello world" } ``` Response adds `text_length`, `chars_typed`, `typos_made`. ### `POST /api/v1/primitive/{page_id}/paste-text` Clipboard paste — fast, use for long strings. ```json { "text": "a very long string…" } ``` Response adds `text_length`, `fumbled`, `delay_before_ms`. ### `POST /api/v1/primitive/{page_id}/press-enter` No body. Response adds `key`. ### `POST /api/v1/primitive/{page_id}/press-tab` ```json { "with_shift": false } ``` `with_shift: true` sends Shift+Tab (focus backwards). Response adds `key`. ### `POST /api/v1/primitive/{page_id}/zoom-page` ```json { "x": 100, "y": 200, "zoom_level": 1.5 } ``` `zoom_level` is optional (default `1`). Response adds `zoom_level` and `position`. ### `POST /api/v1/primitive/{page_id}/reset-zoom` No body. Restores 100% zoom. --- ## Plugins Plugins are discovered at startup from `plugins/` and their routes are generated dynamically, so the list below reflects the plugins currently shipped. The request model of each plugin is built from its declared parameters, so the OpenAPI schema at `/docs` is always authoritative. Route shape: ``` POST /api/v1/{page_id}/plugin/{plugin-path} ``` `{plugin-path}` is the plugin's folder path under `plugins/` (underscores become hyphens), e.g. `archive-check` from `plugins/archive_check/`, or the nested `instagram/open-account` from `plugins/instagram/open_account/`. `GET /api/v1/plugins` reports each plugin's `route` field — use that when building URLs. (Note the different shape: no `ops`/`primitive` segment — `page_id` comes right after `/api/v1`.) Parameter validation happens before execution; a violation returns `400` with the plugin's own message, e.g. `"paste_probability must be between 0.0 and 1.0"`. ### `GET /api/v1/plugins` Lists every registered plugin with its parameter schema. No auth. ```json { "success": true, "page_id": null, "count": 5, "plugins": [ { "name": "archive-check", "description": "Archive Check", "methods": ["POST"], "bg_color": ["#c2601f", "#ff9924"], "parameters": [ { "name": "url", "type": "url", "label": "URL to Check", "required": true, "default": null, "placeholder": "https://example.com", "options": [], "help_text": "URL to check for archived copies on archive.is" } ] } ] } ``` Parameter `type` values map to JSON types: `text`, `url`, `textarea`, `select` → string; `number` → float; `checkbox` → boolean. ### `archive-check` `POST /api/v1/{page_id}/plugin/archive-check` — is a URL archived on archive.is? | Param | Type | Required | Default | Notes | |---|---|---|---|---| | `url` | url | yes | — | URL to look up | | `direct_nav_probability` | number | no | `0.5` | 0.0–1.0; chance of hitting `archive.is/` directly instead of using the search form | | `paste_probability` | number | no | `0.9` | 0.0–1.0; chance of pasting rather than typing the URL | ```json { "success": true, "page_id": "scraper", "url": "https://example.com", "is_archived": true, "wtf": false, "method": "direct_nav" } ``` `wtf` is `true` when the page's heuristics disagree with each other — treat the `is_archived` value as unreliable in that case. `method` is `direct_nav` or `search_form`. ### `archive-save` `POST /api/v1/{page_id}/plugin/archive-save` — submit a URL to archive.is. | Param | Type | Required | Default | Notes | |---|---|---|---|---| | `url` | url | no* | — | *Required unless `from_check` is true | | `from_check` | checkbox | no | `false` | Continue from the not-found page left by `archive-check` instead of navigating | Response: `url`, `from_check`, `current_url`, `message`. ### `archive-get` `POST /api/v1/{page_id}/plugin/archive-get` — open the newest archived snapshot and read it. | Param | Type | Required | Default | Notes | |---|---|---|---|---| | `url` | url | no* | — | *Required unless `from_check` is true | | `from_check` | checkbox | no | `false` | Reuse the result list already on the page (skip the search) | Response: `url`, `archive_url`, `page_title`, `page_content`, `message`. ### `nytimes-search` `POST /api/v1/{page_id}/plugin/nytimes-search` — search nytimes.com, or go straight to a nytimes.com URL. | Param | Type | Required | Default | Notes | |---|---|---|---|---| | `query` | text | yes | — | Search term, or a full nytimes.com URL | | `direct_access_probability` | number | no | `0.5` | 0.0–1.0; direct navigation vs. search form | | `paste_probability` | number | no | `0.8` | 0.0–1.0; paste vs. type | Response: `method`, `url`, `query`, `page_url`, `message`. ### `detect-bounding-box` `POST /api/v1/{page_id}/plugin/detect-bounding-box` — screenshot the page and ask Gemini vision where an object is. | Param | Type | Required | Notes | |---|---|---|---| | `description` | text | yes | e.g. `"search button"`, `"login form"`, `"logo"` | ```json { "success": true, "page_id": "scraper", "description": "search button", "bounding_box": [120, 340, 180, 420], "center_x": 380, "center_y": 150, "tokens_used": 1834, "costs_usd": 0.0004, "duration_seconds": 2.71 } ``` Feed `center_x` / `center_y` to `click-position`. ### `instagram/open-account` `POST /api/v1/{page_id}/plugin/instagram/open-account` — search an Instagram handle and open its profile (search icon → search bar → handle → first profile result). | Param | Type | Required | Notes | |---|---|---|---| | `handle` | text | yes | With or without leading `@`; letters/digits/dots/underscores only | | `paste_probability` | number | no | 0.0–1.0, chance of pasting instead of typing (default 0.3) | ```json { "success": true, "handle": "circoloarcibrigante", "opened_handle": "circoloarcibrigante", "current_handle": "circoloarcibrigante", "current_state": "profile", "result_href": "/circoloarcibrigante/", "input_method": "type", "navigated_home": false, "page_url": "https://www.instagram.com/circoloarcibrigante/", "message": "Opened Instagram profile 'circoloarcibrigante'" } ``` `current_handle` / `current_state` are re-classified from the live page after the navigation, so the client can verify the search actually landed on the requested account (a fuzzy search can open a different handle, e.g. `giopecc` → `giopeccc`). Instagram plugins live under `plugins/instagram/` with shared helpers and the post-overlay guard (a post dialog opened on top of the page is closed before clicking anything behind it). ### `instagram/state` `GET /api/v1/{page_id}/plugin/instagram/state` — observe where the browser is on instagram.com without touching anything (no navigation, no clicks). The state is derived from the live DOM on every call; nothing is stored between calls. Takes no parameters. Returns: ```json { "success": true, "state": "post_open", // feed | profile | post_open | unknown "page_url": "https://www.instagram.com/p/Dc85geACphL/?img_index=1", "handle": null, // profile / post_open "author": "yuliana_prokhorenko", // post_open: author of the open post "shortcode": "Dc85geACphL", // post_open "media_type": "photo", // photo | video (post_open) "liked": false, // post_open: true when "Unlike" is rendered "carousel": { // post_open, carousels only "img_index": 1, "has_previous": false }, "posts_in_view": { // feed: {"articles": N}, profile: {"grid_thumbnails": N} "articles": 4 }, "message": "Instagram state: post_open" } ``` `unknown` states additionally carry a `hint` field with the suggested next move. This endpoint is the bootstrap of the chain: run it first, then pick the next plugin from `state`. ### `instagram/get-posts-profile` `GET /api/v1/{page_id}/plugin/instagram/get-posts-profile` — list the grid posts in view on the currently open profile, in DOM order (newest first, pinned posts in front). Precondition: state must be `profile`. Optional query param `shortcode`: when present, the response contains only the matching post (`success=false` with `error="shortcode_not_in_view"` if it is not in view). Every post carries an exact `selector`, so the client can open one by feeding it straight into primitive `click-element`. ```json { "success": true, "handle": "circoloarcibrigante", "count": 2, "posts": [ { "index": 0, "shortcode": "Dcoj3RZkQ_B", "href": "/circoloarcibrigante/p/Dcoj3RZkQ_B/", "selector": "a[href=\"/circoloarcibrigante/p/Dcoj3RZkQ_B/\"]", "media_type": "photo", "pinned": true, "caption": "momenti felici in ordine sparso" }, { "index": 1, "shortcode": "Dc85geACphL", "href": "/circoloarcibrigante/p/Dc85geACphL/", "selector": "a[href=\"/circoloarcibrigante/p/Dc85geACphL/\"]", "media_type": "carousel", "pinned": false, "caption": null } ], "page_url": "https://www.instagram.com/circoloarcibrigante/", "message": "2 posts in view on 'circoloarcibrigante'" } ``` `media_type` is `photo | carousel | video | reel` (badge svg `aria-label`s); `caption` comes from the thumbnail img alt, which Instagram replaces with `"Photo by on "` when the post has no caption. Only the posts rendered in the grid are listed — scroll the page and call again to see more. ### `instagram/scroll-carousel` `POST /api/v1/{page_id}/plugin/instagram/scroll-carousel` — step the open post dialog one media left/right. Precondition: state must be `post_open`. | Param | Type | Required | Notes | |---|---|---|---| | `direction` | options | yes | `right` (next media) or `left` (previous media) | ```json { "success": true, "direction": "right", "moved": true, "position": 2, "total_media": 12, "current_media_type": "img", "has_previous": true, "has_next": true, "page_url": "https://www.instagram.com/p/Dcoj3RZkQ_B/?img_index=2", "message": "Media 2/12 (img)" } ``` - `total_media` counts every media in the carousel (imgs + videos), read from the dots tray; single-media posts report `1` - `position` is 1-based (the active dot, falling back to the url's `img_index`) - `current_media_type` is `img | video` — Instagram only renders the current slide's element, so a visible `