rushabh/SimpleScraper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prune profile payload for options thesis

Browse Source
This commit is contained in:
Rushabh Gosar
2025-12-29 12:27:30 -08:00
parent 68805ed80a
commit c01a98abce
3 changed files with 273 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
AGENTS.md
View File

@@ -22,8 +22,7 @@
- Behavior:
- Loads `https://finance.yahoo.com/quote/<SYMBOL>/` with Playwright.
- Pulls the embedded SvelteKit payloads (quoteSummary, quote, quoteType, ratings, recommendations).
- Parses rendered DOM for recent news and the ticker news summary.
- Returns company profile, key stats, earnings, analyst data, performance, and news in JSON.
- Returns a pruned JSON with valuation, profitability, growth, financial strength, cashflow, ownership, analyst, earnings, and performance summaries.
## Guard Rails
- Run local 10-cycle validation (4 stocks x 4 expiries) before any deploy or push.
@@ -45,12 +44,12 @@
- Validate: `python scripts/test_profile_cycles.py --base-url http://127.0.0.1:9777/profile --runs 8`
## Update Log (2025-12-28)
- Added `/profile` endpoint backed by SvelteKit payload parsing (quoteSummary, quote, quoteType, ratings, recommendations) plus rendered news extraction.
- Response now includes company profile, key stats, earnings, analyst trends, performance overview, ticker news summary, and recent news items.
- Validation added to ensure quote data matches the requested symbol, with issues reported in `validation`.
- Added `/profile` endpoint backed by SvelteKit payload parsing (quoteSummary, quote, quoteType, ratings, recommendations).
- `/profile` response trimmed to focus on valuation, profitability, growth, financial strength, cashflow, ownership, analyst, earnings, and performance summaries.
- Validation ensures quote data matches the requested symbol, with issues reported in `validation`.
- Issue encountered: existing server instance bound to port 9777 without `/profile`, resolved by restarting the service with the updated script.
- Tests executed (local):
- `.\venv\Scripts\python.exe scripts\test_profile_cycles.py --runs 8 --timeout 180`
- `.\venv\Scripts\python.exe scripts/test_profile_cycles.py --runs 8 --timeout 180`
- `.\venv\Scripts\python.exe scripts\test_cycles.py --base-url http://127.0.0.1:9777/scrape_sync`
- Tests executed (docker):
- `docker build -t rushabhtechie/yahoo-scraper:latest .`