You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1.6 KiB
1.6 KiB
Challenge Deliverables
- Architecture diagram
- saved as a PNG
- Explain decisions and trade-offs
- Document in README.md
- Terraform repo scaffolding out the service
Air Traffic Metrics
- Ingest raw data into S3
- S3 Bucket
- Bucket best practices
- Buckets aren't public
- Only
loadlambda can add objects - Only Athena can read objects
- ????
- Bucket best practices
- S3 Bucket
- Database of my choice
- Athena?
- Less up-front configuration involved. I hope.
- AWS wants to lock you into their ecosystem so it'll be cheaper.
- In the future, we can switch from CSV to JSON or Parquet or mix and match without any overhead.
- Aurora (Postgresql)
- Aurora (Mysql)
- Athena?
- API to interact with the data
- API Gateway
- Policy that grants it lambda invocation rights
/loadendpoint- Python lambda to upload data to bucket
- Example data: https://zenodo.org/records/5377831
- Python lambda to upload data to bucket
/metricsendpoint- Python lambda that returns the following:
- row_count: number of rows in table for full data set
- last_transponder_seen_at: maximum value for
lastseen - most_popular_destination: most seen value for destination
- count_of_unique_transponders: unique count of the icao24 field
- Execution role that grants it access to athena database
- Python lambda that returns the following:
- API Gateway
- Frontend to display the data
- Python lambda that renders an HTML page that displays above metrics