|
|
@ -1,9 +1,22 @@
|
|
|
|
|
|
|
|
## Justifications
|
|
|
|
|
|
|
|
- Athena
|
|
|
|
|
|
|
|
- Less up-front configuration involved. I hope.
|
|
|
|
|
|
|
|
- AWS wants to lock you into their ecosystem so it'll be cheaper.
|
|
|
|
|
|
|
|
- In the future, we can switch from CSV to JSON or Parquet or mix and match without any overhead.
|
|
|
|
|
|
|
|
- API Gateway
|
|
|
|
|
|
|
|
- Rate limiting and caching built-in
|
|
|
|
|
|
|
|
- Abstracts interfacing with the API away from raw lambda invocation URLs
|
|
|
|
|
|
|
|
- Can be mapped to a domain
|
|
|
|
|
|
|
|
- Athena
|
|
|
|
|
|
|
|
- Less up-front configuration involved. I hope.
|
|
|
|
|
|
|
|
- AWS wants to lock you into their ecosystem so it'll be cheaper.
|
|
|
|
|
|
|
|
- In the future, we can switch from CSV to JSON or Parquet or mix and match without any overhead.
|
|
|
|
## Challenge Deliverables
|
|
|
|
## Challenge Deliverables
|
|
|
|
- [ ] Architecture diagram
|
|
|
|
- [X] Architecture diagram
|
|
|
|
- [ ] saved as a PNG
|
|
|
|
- [X] saved as a PNG
|
|
|
|
- [ ] Explain decisions and trade-offs
|
|
|
|
- [X] Explain decisions and trade-offs
|
|
|
|
- [ ] Document in README.md
|
|
|
|
- [X] Document in README.md
|
|
|
|
- [ ] Terraform repo scaffolding out the service
|
|
|
|
- [X] Terraform repo scaffolding out the service
|
|
|
|
## Air Traffic Metrics
|
|
|
|
## Air Traffic Metrics
|
|
|
|
- [ ] Ingest raw data into S3
|
|
|
|
- [ ] Ingest raw data into S3
|
|
|
|
- [X] S3 Bucket
|
|
|
|
- [X] S3 Bucket
|
|
|
@ -19,13 +32,13 @@
|
|
|
|
- In the future, we can switch from CSV to JSON or Parquet or mix and match without any overhead.
|
|
|
|
- In the future, we can switch from CSV to JSON or Parquet or mix and match without any overhead.
|
|
|
|
- [ ] Aurora (Postgresql)
|
|
|
|
- [ ] Aurora (Postgresql)
|
|
|
|
- [ ] Aurora (Mysql)
|
|
|
|
- [ ] Aurora (Mysql)
|
|
|
|
- [ ] API to interact with the data
|
|
|
|
- [X] API to interact with the data
|
|
|
|
- [ ] API Gateway
|
|
|
|
- [X] API Gateway
|
|
|
|
- [ ] Policy that grants it lambda invocation rights
|
|
|
|
- [X] Policy that grants it lambda invocation rights
|
|
|
|
- [ ] `/load` endpoint
|
|
|
|
- [X] `/load` endpoint
|
|
|
|
- [ ] Python lambda to upload data to bucket
|
|
|
|
- [ ] Python lambda to upload data to bucket
|
|
|
|
- [ ] Example data: https://zenodo.org/records/5377831
|
|
|
|
- [ ] Example data: https://zenodo.org/records/5377831
|
|
|
|
- [ ] `/metrics` endpoint
|
|
|
|
- [X] `/metrics` endpoint
|
|
|
|
- [ ] Python lambda that returns the following:
|
|
|
|
- [ ] Python lambda that returns the following:
|
|
|
|
- [ ] row_count: number of rows in table for full data set
|
|
|
|
- [ ] row_count: number of rows in table for full data set
|
|
|
|
- [ ] last_transponder_seen_at: maximum value for `lastseen`
|
|
|
|
- [ ] last_transponder_seen_at: maximum value for `lastseen`
|
|
|
@ -33,4 +46,4 @@
|
|
|
|
- [ ] count_of_unique_transponders: unique count of the icao24 field
|
|
|
|
- [ ] count_of_unique_transponders: unique count of the icao24 field
|
|
|
|
- [ ] Execution role that grants it access to athena database
|
|
|
|
- [ ] Execution role that grants it access to athena database
|
|
|
|
- [ ] Frontend to display the data
|
|
|
|
- [ ] Frontend to display the data
|
|
|
|
- [ ] Python lambda that renders an HTML page that displays above metrics
|
|
|
|
- [X] Python lambda that renders an HTML page that displays above metrics
|
|
|
|