Added a Readme and License file
This commit is contained in:
parent
687513725c
commit
556006d8c6
21
LICENSE
Normal file
21
LICENSE
Normal file
@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Jarian Cottingham
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included
|
||||
in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
|
||||
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
|
||||
EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
|
||||
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
|
||||
OTHERWISE ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
|
||||
USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
109
README.md
Normal file
109
README.md
Normal file
@ -0,0 +1,109 @@
|
||||
# NASA Image Downloader
|
||||
|
||||
Download images from the NASA Image & Video Library API into a local folder structure,
|
||||
store each image’s metadata in a JSON file, and automatically resume where the
|
||||
last run stopped. The script is idempotent and runs safely as a nightly job or
|
||||
as a manual command.
|
||||
|
||||
## Description
|
||||
|
||||
- Queries NASA’s public “Image and Video Library” API for all images between a
|
||||
start and an end date.
|
||||
- Downloads each image to an `images/` directory.
|
||||
- Saves the raw API response for each item to a companion
|
||||
`index‑<nasa_id>.json` file in `metadata/`.
|
||||
- Keeps a `last_run.txt` file with the date of the most recent successful run;
|
||||
subsequent executions automatically pick up from that date.
|
||||
- Skips files that already exist locally, so the script can be called multiple
|
||||
times without re‑downloading anything.
|
||||
- Optional time‑budget: stops after a configurable maximum run time (2 h by
|
||||
default).
|
||||
- Graceful error handling – printing failures to STDERR but continuing the
|
||||
entire run.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.7 or later (the shebang is `/usr/bin/env python3`).
|
||||
- Python packages:
|
||||
- `requests`
|
||||
- `tqdm` (optional; if missing the script falls back to a plain list)
|
||||
|
||||
```bash
|
||||
pip install requests tqdm
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python downloader.py [--start YYYY-MM-DD] [--end YYYY-MM-DD] [--output BASE_DIR]
|
||||
```
|
||||
|
||||
| Argument | Description |
|
||||
|----------|-------------|
|
||||
| `--start` | Start date (inclusive). If omitted, the script reads the date from
|
||||
`last_run.txt`; if that file does not exist it defaults to **three days ago**. |
|
||||
| `--end` | End date (inclusive). Defaults to **today**. |
|
||||
| `--output` | Base directory where `images/` and `metadata/` folders will be
|
||||
created. Default is the current working directory. |
|
||||
|
||||
The script prints progress to the console and records the last successful
|
||||
end date in `last_run.txt` (or in the chosen output directory).
|
||||
|
||||
## Examples
|
||||
|
||||
Below are valid calls that demonstrate typical use‑cases.
|
||||
|
||||
```bash
|
||||
# 1. Default run – uses last_run.txt for the start date (or defaults to
|
||||
# 3 days ago) and downloads everything up to today.
|
||||
python downloader.py
|
||||
|
||||
# 2. Override date range – gather images created in the first week of July 2023.
|
||||
python downloader.py --start 2023-07-01 --end 2023-07-07
|
||||
|
||||
# 3. Recent images only – start is yesterday, end is today.
|
||||
python downloader.py --start $(date -u +%Y-%m-%d --date='-1 day') \
|
||||
--end $(date -u +%Y-%m-%d)
|
||||
|
||||
# 4. Output everything under a specific directory tree.
|
||||
python downloader.py --output /Users/me/Nasa
|
||||
|
||||
# 5. Custom state directory: images and metadata will live inside `/tmp/nasa_imgs/`.
|
||||
python downloader.py --start 2023-06-20 --output /tmp/nasa_imgs
|
||||
|
||||
# 6. Combine date range and custom output base.
|
||||
python downloader.py --start 2023-01-01 \
|
||||
--end 2023-01-05 \
|
||||
--output /Projects/nasa_imgs
|
||||
```
|
||||
|
||||
## Output Structure
|
||||
|
||||
After a successful run:
|
||||
|
||||
```
|
||||
images/
|
||||
├── <high‑res‑image‑1>.jpg
|
||||
├── <high‑res‑image‑2>.png
|
||||
└── ...
|
||||
|
||||
metadata/
|
||||
├── index-XYZ123.json
|
||||
├── index-ABC456.json
|
||||
└── ...
|
||||
|
||||
last_run.txt
|
||||
```
|
||||
|
||||
- Each `index‑<nasa_id>.json` contains the full API response for the image.
|
||||
- The `images` folder holds the highest‑resolution image that NASA offers
|
||||
for each item.
|
||||
|
||||
## Author
|
||||
|
||||
*Created by the NASA Open‑Source Initiative / community contributors.*
|
||||
|
||||
## License
|
||||
|
||||
This project is released under the MIT License. See `LICENSE` for details.
|
||||
```
|
||||
0
wrapper.sh
Normal file → Executable file
0
wrapper.sh
Normal file → Executable file
Loading…
x
Reference in New Issue
Block a user