b41a43bf89
CI: add generalized workflow
2026-07-05 02:46:40 +00:00
0fa94d3ee3
feat: enhance logging for AI service error diagnosis
...
Added comprehensive logging to diagnose 'Expecting value: line 1 column 1 (char 0)' errors in AI service responses. The changes include detailed logging of AI requests/responses, cache operations, and article processing steps to better identify when the AI service returns empty or invalid responses.
2026-02-02 12:28:12 -06:00
271ede7845
Enhance AI processor error logging for better debugging
2026-02-02 10:56:10 -06:00
876cfddce7
Fix AI processor error handling for empty AI service responses
2026-02-02 10:52:34 -06:00
e29e5dbc76
Remove file extension filtering - all files in articles directory are valid articles
2026-02-02 10:38:35 -06:00
492b30a713
Fix cache manager to properly handle empty cache files and add diagnostic logging
2026-02-02 10:35:55 -06:00
35d8f44ee8
Fix cache logic bug that caused all files to be incorrectly marked as processed
2026-02-02 10:17:06 -06:00
218cad82cc
Fix cache file creation and validation issues - ensure cache file is properly created and handled
2026-02-02 10:02:59 -06:00
b8228d8e07
Fix pathing issue for article directory - use relative path ../scraper/articles
2026-02-02 09:55:24 -06:00
ad2ef2a87b
Remove cache file from version control, fix cache initialization logic
2026-02-02 09:50:42 -06:00
e9c66da456
Fix AI processor cache initialization issue and create empty cache file
2026-02-02 09:47:35 -06:00
6ce68a9037
Add key to api in docker
2026-02-02 09:37:06 -06:00
abb63db4b7
enhancement: Make article processor more robust with multiple path support and fallback file format detection
2026-02-02 09:31:14 -06:00
ba8f24fcbd
fix: Update scraper directory path for Docker container compatibility and add debugging
2026-02-02 09:27:02 -06:00
0b82bb13c4
fix: Handle missing cache_stats key in logging to prevent KeyError
2026-02-02 09:24:02 -06:00
e7b31a510a
fix: Complete removal of relative imports to resolve Docker container issues
2026-02-02 09:20:44 -06:00
5c47f6c235
fix: Resolve Docker import errors and relative import issues
2026-02-02 09:05:53 -06:00
e641a935a7
fix: Remove logging from requirements.txt due to Docker build error
2026-02-02 08:58:40 -06:00
714f5380e7
feat: Implement AI processor with fact extraction capabilities for articles from scraper
2026-02-02 08:53:58 -06:00
cb6c42fa65
gutting old ai_processor code that's not usful
2026-02-02 08:34:48 -06:00
9ab3e459e5
fix logging path in embedding pipe
2026-02-02 08:30:22 -06:00
68704b6edd
Add API key authentication for AI service connections
2026-02-02 04:29:50 -06:00
f6ded4fc14
Uncomment ai_processor service in docker-compose.yml to enable AI processing
2026-02-02 01:36:38 -06:00
dfa729f204
Fix embedding Dockerfile by removing unnecessary build dependencies
2026-02-02 01:28:26 -06:00
1dd9337c39
Fix embedding service by removing unused openai import and cleaning up dependencies
2026-02-02 01:23:14 -06:00
da03b3e847
Clean up requirements files and remove unnecessary dependencies including PyTorch
2026-02-01 21:11:22 -06:00
157e4541fc
Implement enhanced cache system with batch processing and two-phase processing approach
2026-02-01 21:03:00 -06:00
56231fa354
feat: Implement automated cron job setup for embedding pipeline and clean up AI processor fact extraction logic
...
• Created automated setup_embedding_cron_auto.sh script that fully configures cron jobs without manual intervention
• Enhanced embedding pipeline logging and error handling
• Simplified AI processor to focus on core fact extraction functionality
• Added proper logging to all scripts for better monitoring
2026-02-01 20:39:07 -06:00
6eb5e05b1e
get rid of cron.logs
2026-02-01 15:08:35 -06:00
8734ef9fcc
fix git ignore
2026-02-01 15:00:53 -06:00
211cf0b89e
added embedder to docker compose file and turned into cron job
2026-02-01 14:52:33 -06:00
91860211ad
Install dependencies and update requirements.txt for ChromaDB query functionality
2026-02-01 10:21:00 -06:00
c8a712d776
feat: implement enhanced data pipeline with Prometheus telemetry and Grafana integration
2026-01-31 23:56:06 -06:00
5e52142909
Add article server deployment to docker-compose and remove scraper service
...
Added article-server service to docker-compose.yml that builds from articleServer directory and maps articles volume from /home/user/StockDocs/scraper/articles. Removed the scraper service as it's no longer needed. The article server will now serve articles on port 5008.
2026-01-31 23:22:46 -06:00
c6f3cb3e65
Update n8n webhook URL to production endpoint
2026-01-31 20:52:53 -06:00
64e81869ea
Restore /facts endpoint and maintain all MCP Server functionality
2026-01-31 13:10:59 -06:00
ec756240b1
Implement all MCP Server endpoints from OpenAPI specification
2026-01-31 13:09:31 -06:00
1a62e2bd9b
cleaned up mcp server andn added /facts endpoint
2026-01-31 13:05:23 -06:00
0453faccb4
Add comprehensive agent.md overview file for repository projects
2026-01-31 12:36:08 -06:00
cc192cf566
Implement article filtering to avoid reprocessing already downloaded articles
2026-01-31 12:06:37 -06:00
1cbeb55463
Fix filename sanitization to resolve Linux permission denied errors
2026-01-31 11:53:32 -06:00
7808922aac
Remove U.S. News – Money RSS feed that was causing connection issues
2026-01-31 11:45:44 -06:00
e23b4c2ac9
Add enhanced error handling for 'Remote end closed connection' errors in RSS feed parsing
2026-01-31 11:44:08 -06:00
e975350701
Add explicit Firefox path handling for /usr/bin/firefox to fix PATH issues
2026-01-31 11:38:48 -06:00
ea99b69e87
Add better error handling for Firefox binary not found error
2026-01-31 11:34:50 -06:00
341cbde833
Minor logging update - existing logging already shows proper article counts
2026-01-31 11:33:49 -06:00
d5625605ad
Add more robust timeout handling for RSS feed parsing to prevent hanging feeds
2026-01-31 11:27:14 -06:00
a07c637984
Remove problematic NASDAQ feed that was holding up threads
2026-01-31 11:24:59 -06:00
4ee338ae4d
Optimize RSS feed processing with parallel parsing and improved error handling
2026-01-31 11:23:25 -06:00
03b728d172
Improve logging in scraper with timestamps and focused new file notifications
2026-01-31 11:16:05 -06:00
