# JuggerBot – WebShot Tor Crawler (Stable Beta)

JuggerBot is a desktop web screenshot crawler designed for full-page rendered capture, controlled crawling, and structured output.

It combines a Tkinter UI, Playwright rendering, httpx crawling, and optional Tor routing into a single self-contained tool for capturing and documenting web targets.

---

## Core Capabilities

- Full-page rendered screenshots (HTML → PNG)
- Playwright-powered browser capture (Chromium)
- Tor SOCKS5 proxy routing
- Optional Tor NEWNYM identity rotation
- Same-domain crawling with configurable depth
- Optional file discovery and download scraping
- JSON run logging
- Portable HTML report generation
- Stop-safe capture (interrupts active screenshot execution)

---

## Architecture Overview

JuggerBot splits responsibility across layers:

- UI (Tkinter): control panel and runtime feedback
- Playwright: rendered page capture engine
- httpx: crawl and download handling
- stem: Tor control and identity rotation
- Reporter: JSON and HTML output
- File system: screenshots, logs, downloads, reports

---

## Project Layout

webshot_tor_app/
├── webshot_app.py
├── .config.json
├── requirements.txt
├── README.md
├── app/
│   ├── config_manager.py
│   ├── crawler.py
│   ├── reporter.py
│   ├── screenshotter.py
│   ├── tor_utils.py
│   └── utils.py
├── screenshots/
├── downloads/
├── logs/
└── reports/

---

## Installation

Create a virtual environment (recommended):

    python -m venv venv
    source venv/bin/activate

Install dependencies:

    pip install -r requirements.txt

Install Playwright browser:

    python -m playwright install chromium

---

## Running

    python webshot_app.py

---

## Tor Configuration

Typical local Tor setup:

    SocksPort 9050
    ControlPort 9051
    HashedControlPassword <generated hash>
    CookieAuthentication 0

Default values used by JuggerBot:

- SOCKS Proxy: socks5://127.0.0.1:9050
- Control Host: 127.0.0.1
- Control Port: 9051

If a password is set, it must be entered in the UI or .config.json.

---

## Output Structure

dist/
├── screenshots/
├── downloads/
├── logs/
└── reports/

Screenshots:
Rendered full-page PNG captures.

Downloads:
Optional scraped files from crawled pages.

Logs:
Structured JSON output of each run.

Reports:
Human-readable HTML reports.

---

## Report Behavior (Important)

HTML reports use relative paths, not absolute paths.

Example:

    ../screenshots/file.png
    ../downloads/file.zip

This ensures:

- Reports are portable
- No manual path editing is required
- Reports open correctly from the /reports directory

---

## Stop Behavior (Stable Beta Fix)

The Stop button:

- Interrupts active Playwright screenshot capture
- Terminates the capture process safely
- Prevents UI lockups from blocking browser calls

Behavior notes:

- Stop is immediate during screenshot capture
- Crawl and download loops stop on next iteration
- In-flight network requests may complete before stopping

---

## Packaging (PyInstaller)

Build:

    pyinstaller --noconfirm --onefile --windowed webshot_app.py

Playwright Requirement:

Playwright must be able to locate its browser binaries at runtime.

Ensure the application sets:

    os.environ["PLAYWRIGHT_BROWSERS_PATH"] = os.path.expanduser("~/.cache/ms-playwright")

This prevents errors such as:

    Executable doesn't exist at /tmp/_MEI.../playwright/...

---

## Known Limitations

- Extremely long pages may exceed memory or rendering limits
- Infinite scroll pages may not fully render
- Some sites block Tor exit nodes
- Tor NEWNYM is rate-limited by Tor itself
- Playwright introduces significant runtime overhead

---

## Configuration File

JuggerBot stores settings in:

    .config.json

This file should remain alongside the executable or script.

---

## Notes

- This is a stable beta, not a final release
- Core functionality is operational
- Additional refinement is expected in crawling, reporting, and packaging

---

## Summary

JuggerBot evolved from a simple HTML-to-PNG tool into a structured capture system:

- UI-driven crawler
- Rendered screenshot engine
- Tor-aware routing
- Structured logging and reporting

Built for control, inspection, and repeatable capture workflows.