Scalable data streaming with Amazon kinesis design and secure highly available, cost-effective data streaming applications with Amazon kinesis
This practical guide takes a hands-on approach to implementation and associated methodologies to have you up and running with all that Amazon Kinesis has to offer. You'll work with use cases and practical examples to be able to ingest, process, analyze, and stream real-time data in no time.
| Otros Autores: | |
|---|---|
| Formato: | Libro electrónico |
| Idioma: | Inglés |
| Publicado: |
Birmingham, England ; Mumbai :
Packt Publishing
[2021]
|
| Materias: | |
| Ver en Biblioteca Universitat Ramon Llull: | https://discovery.url.edu/permalink/34CSUC_URL/1im36ta/alma991009631703106719 |
Tabla de Contenidos:
- Cover
- Title Page
- Copyright and Credits
- Contributors
- Table of Contents
- Preface
- Section 1: Introduction to Data Streaming and Amazon Kinesis
- Chapter 1: What Are Data Streams?
- Introducing data streams
- Sources of data
- The value of real-time data in analytics
- Decoupling systems
- Challenges associated with distributed systems
- Transactions per second
- Scaling
- Latency
- Fault tolerance/high availability
- Overview of messaging concepts
- Overview of core messaging components
- Messaging concepts
- Examples of data streaming
- Application log processing
- Internet of Things
- Real-time recommendations
- Video streams
- Summary
- Further reading
- Chapter 2: Messaging and Data Streaming in AWS
- Amazon Kinesis Data Streams (KDS)
- Encryption, authentication, and authorization
- Producing and consuming records
- Data delivery guarantees
- Integration with other AWS services
- Monitoring
- Amazon Kinesis Data Firehose (KDF)
- Encryption, authentication, and authorization
- Monitoring
- Producers
- Delivery destinations
- Transformations
- Amazon Kinesis Data Analytics (KDA)
- Amazon KDA for SQL
- Amazon Kinesis Data Analytics for Apache Flink (KDA Flink)
- Amazon Kinesis Video Streams (KVS)
- Amazon Simple Queue Service (SQS)
- Amazon Simple Notification Service (SNS)
- Amazon SNS integrations with other AWS services
- Encryption at rest
- Amazon MQ for Apache ActiveMQ
- IoT Core
- Device software
- Control services
- Analytics services
- Amazon Managed Streaming for Apache Kafka (MSK)
- Apache Kafka
- Amazon MSK
- Amazon EventBridge
- Service comparison summary
- Summary
- Chapter 3: The SmartCity Bike-Sharing Service
- The mission for sustainable transportation
- SmartCity new mobile features
- SmartCity data pipeline
- SmartCity data lake.
- SmartCity operations and analytics dashboard
- SmartCity video
- The AWS Well-Architected Framework
- Summary
- Further reading
- Section 2: Deep Dive into Kinesis
- Chapter 4: Kinesis Data Streams
- Technical requirements
- Discovering Amazon Kinesis Data Streams
- Creating streams and shards
- Creating a stream producer application
- Creating a stream consumer application
- Data pipelines with Amazon Kinesis Data Streams
- Data pipeline design (simple)
- Data pipeline design (intermediate)
- Data pipeline design (full design)
- Designing for scalable and reliable analytics pipelines
- Monitoring and scaling with Amazon Kinesis Data Streams
- X-Ray tracing with Amazon Kinesis Data Streams
- Scaling up with Amazon Kinesis Data Streams
- Securing Amazon Kinesis Data Streams
- Implementing least-privilege access
- Summary
- Further reading
- Chapter 5: Kinesis Firehose
- Technical requirements
- Setting up the AWS account
- Using a local development environment
- Using an AWS Cloud9 development environment
- Code examples
- Discovering Amazon Kinesis Firehose
- Understanding KDF delivery streams
- Understanding encryption in KDF
- Using data transformation in KDF with a Lambda function
- Understanding delivery stream destinations
- Amazon S3
- Amazon Redshift
- Amazon Elasticsearch Service
- Splunk destination
- HTTP endpoint destination
- Understanding data format conversion in KDF
- Deserialization
- Schema
- Serializer
- Data format conversion errors
- Understanding monitoring in KDF
- Use-case example - Bikeshare station data pipeline with KDF
- Steps to recreate the example
- Summary
- Further reading
- Chapter 6: Kinesis Data Analytics
- Technical requirements
- AWS account setup
- AWS CDK
- Java and Java IDE
- Code examples
- Discovering Amazon KDA.
- Working on SmartCity bike share analytics use cases
- Creating operational insights using SQL Engine
- Core concepts and capabilities
- Creating operational insights using Apache Flink
- Options for running Flink applications in AWS Cloud
- Flink applications on KDA
- Building bike ride analytic applications
- Setting up a producer application
- Building a KDA SQL application
- Building a KDA Flink application
- Monitoring KDA applications
- Summary
- Further reading
- Blogs
- Workshops
- Chapter 7: Amazon Kinesis Video Streams
- Technical requirements
- AWS account setup
- Using a local development environment
- Code examples
- Understanding video fundamentals
- Containers
- Codecs
- Discovering Amazon Kinesis video streams WebRTC
- Core concepts and connection patterns
- Creating a signaling channel
- Establishing a connection
- Discovering Amazon KVS
- Key components of KVS
- Stream
- Kinesis producer
- Consuming
- Creating a stream
- Producing
- Integration with Rekognition
- Building video-enabled applications with KVS
- Summary
- Further reading
- Section 3: Integrations
- Chapter 8: Kinesis Integrations
- Technical requirements
- AWS account setup
- AWS CLI
- Kinesis Data Generator
- Code examples
- Amazon services that can produce data to send to Kinesis
- Amazon Connect
- Amazon Aurora database activity
- DynamoDB activity
- Processing Kinesis data with Apache Spark
- Amazon services that consume data from Kinesis
- Serverless data lake
- Amazon services that transform Kinesis data
- Routing events with EventBridge
- Third-party integrations with Kinesis
- Splunk
- Summary
- Further reading
- Why subscribe?
- Other Books You May Enjoy
- Index.
