StockDocs/articleServer/README.md

# Article Server

A Flask-based HTTP server that provides access to news articles stored in a directory structure, with time-based filtering and outlet-specific querying capabilities.

## Overview

This server allows you to query news articles from various sources based on:
- Time range (last hour, day, week, month)
- Specific news outlets
- Direct content retrieval by file path

Articles are organized in a nested directory structure where each news outlet has its own subdirectory containing the respective articles.

## Directory Structure

The server expects the following directory structure:
```
scraper/
└── articles/
    ├── Reuters – Business News/
    │   ├── article1.txt
    │   ├── article2.txt
    │   └── ...
    ├── Associated Press – Business/
    │   ├── article1.txt
    │   └── ...
    └── ...
```

## Endpoints

### 1. Get Articles (`/articles`)
Retrieve articles within a specified time range.

**Method:** `GET`
**Parameters:**
- `time_range` (optional): hour, day, week, month (default: hour)
- `outlets` (optional): comma-separated list of news outlet names

**Example:**
```bash
# Get articles from last day for specific outlets
curl "http://localhost:5008/articles?time_range=day&outlets=Reuters – Business News,Associated Press – Business"

# Get all articles from the last hour
curl "http://localhost:5008/articles"
```

**Response:**
```json
{
  "articles": [
    {
      "path": "/path/to/article.txt",
      "name": "article.txt",
      "outlet": "Reuters – Business News",
      "created_at": "2023-10-17T14:30:00"
    }
  ],
  "count": 5,
  "time_range": "hour",
  "outlets": ["Reuters – Business News"]
}
```

### 2. Get Article Content (`/article/content`)
Retrieve the full content of a specific article by file path.

**Method:** `GET`
**Parameters:**
- `path` (required): Absolute path to the article file

**Example:**
```bash
# Get full content of an article
curl "http://localhost:5008/article/content?path=/full/path/to/article.txt"
```

**Response:**
```json
{
  "path": "/full/path/to/article.txt",
  "name": "article.txt",
  "outlet": "Reuters – Business News",
  "content": "Full article content here..."
}
```

### 3. Get Available Outlets (`/outlets`)
List all available news outlets.

**Method:** `GET`
**Example:**
```bash
curl "http://localhost:5008/outlets"
```

**Response:**
```json
{
  "news_outlets": [
    "Reuters – Business News",
    "Associated Press – Business",
    "Financial Times",
    ...
  ],
  "count": 60
}
```

### 4. Health Check (`/health`)
Simple health check endpoint.

**Method:** `GET`
**Example:**
```bash
curl "http://localhost:5008/health"
```

**Response:**
```json
{
  "status": "healthy"
}
```

## Configuration

### Environment Variables

- `ARTICLE_DIR` (optional): Path to the article directory. Defaults to `scraper/articles` if not set.

## Installation

1. Install dependencies:
```bash
pip install -r requirements.txt
```

2. Set the article directory path (optional):
```bash
export ARTICLE_DIR="/path/to/your/articles"
```

3. Run the server:
```bash
python run_server.py
```

Or with Docker:

```bash
docker build -t article-server .
docker run -p 5008:5008 article-server
```

## Usage Examples

### Get recent articles from all outlets:
```bash
curl "http://localhost:5008/articles?time_range=hour"
```

### Get articles from the last day for specific outlets:
```bash
curl "http://localhost:5008/articles?time_range=day&outlets=Reuters – Business News,Associated Press – Business"
```

### Get all available news outlets:
```bash
curl "http://localhost:5008/outlets"
```

### Retrieve full content of a specific article:
```bash
curl "http://localhost:5008/article/content?path=/absolute/path/to/your/article.txt"
```

## Security Notes

- The server validates that all requested file paths are within the configured article directory to prevent directory traversal attacks
- All file paths must be absolute and within the allowed directory structure
- Path parameters are URL decoded for proper handling of special characters

## Requirements

- Python 3.6+
- Flask 2.3.3

## License

This project is licensed under the MIT License.