Learning Elastic Stack 6.0 a beginner's guide to distributed search, analytics, and visualization using Elasticsearch, Logstash, and Kibana
Deliver end-to-end real-time distributed data processing solutions by leveraging the power of Elastic Stack 6.0 About This Book Get to grips with the new features introduced in Elastic Stack 6.0 Get valuable insights from your data by working with the different components of the Elastic stack such a...
| Otros Autores: | , |
|---|---|
| Formato: | Libro electrónico |
| Idioma: | Inglés |
| Publicado: |
Birmingham, England :
Packt
2017.
|
| Edición: | 1st edition |
| Materias: | |
| Ver en Biblioteca Universitat Ramon Llull: | https://discovery.url.edu/permalink/34CSUC_URL/1im36ta/alma991009630703106719 |
Tabla de Contenidos:
- Cover
- Copyright
- Credits
- Disclaimer
- About the Authors
- About the Reviewer
- www.PacktPub.com
- Customer Feedback
- Table of Contents
- Preface
- Chapter 1: Introducing Elastic Stack
- What is Elasticsearch, and why use it?
- Schemaless and document-oriented
- Searching
- Analytics
- Rich client library support and the REST API
- Easy to operate and easy to scale
- Near real time
- Lightning fast
- Fault tolerant
- Exploring the components of Elastic Stack
- Elasticsearch
- Logstash
- Beats
- Kibana
- X-Pack
- Security
- Monitoring
- Reporting
- Alerting
- Graph
- Elastic Cloud
- Use cases of Elastic Stack
- Log and security analytics
- Product search
- Metrics analytics
- Web search and website search
- Downloading and installing
- Installing Elasticsearch
- Installing Kibana
- Summary
- Chapter 2: Getting Started with Elasticsearch
- Using the Kibana Console UI
- Core concepts
- Index
- Type
- Document
- Node
- Cluster
- Shards and replicas
- Mappings and data types
- Data types
- Core datatypes
- Complex datatypes
- Other datatypes
- Mappings
- Creating an index with the name catalog
- Defining the mappings for the type of product
- Inverted index
- CRUD operations
- Index API
- Indexing a document by providing an ID
- Indexing a document without providing an ID
- Get API
- Update API
- Delete API
- Creating indexes and taking control of mapping
- Creating an index
- Creating type mapping in an existing index
- Updating a mapping
- REST API overview
- Common API conventions
- Formatting the JSON response
- Dealing with multiple indices
- Searching all documents in one index
- Searching all documents in multiple indexes
- Searching all documents of a particular type in all indices
- Summary
- Chapter 3: Searching-What is Relevant
- Basics of text analysis.
- Understanding Elasticsearch analyzers
- Character filters
- Tokenizer
- Standard Tokenizer
- Token filters
- Using built-in analyzers
- Standard Analyzer
- Implementing autocomplete with a custom analyzer
- Searching from structured data
- Range query
- Range query on numeric types
- Range query with score boosting
- Range query on dates
- Exists query
- Term query
- Searching from full text
- Match query
- Operator
- minimum_should_match
- Fuzziness
- Match phrase query
- Multi match query
- Querying multiple fields with defaults
- Boosting one or more fields
- With types of multi match queries
- Writing compound queries
- Constant score query
- Bool query
- Combining OR conditions
- Combining conditions AND and OR conditions
- Adding NOT conditions
- Summary
- Chapter 4: Analytics with Elasticsearch
- The basics of aggregations
- Bucket aggregations
- Metric aggregations
- Matrix aggregations
- Pipeline aggregations
- Preparing data for analysis
- Understanding the structure of data
- Loading the data using Logstash
- Metric aggregations
- Sum, average, min, and max aggregations
- Sum aggregation
- Average aggregation
- Min aggregation
- Max aggregation
- Stats and extended stats aggregations
- Stats aggregation
- Extended stats Aggregation
- Cardinality aggregation
- Bucket aggregations
- Bucketing on string data
- Terms aggregation
- Bucketing on numeric data
- Histogram aggregation
- Range aggregation
- Aggregations on filtered data
- Nesting aggregations
- Bucketing on custom conditions
- Filter aggregation
- Filters aggregation
- Bucketing on date/time data
- Date Histogram aggregation
- Creating buckets across time
- Using a different time zone
- Computing other metrics within sliced time intervals
- Focusing on a specific day and changing intervals.
- Bucketing on geo-spatial data
- Geo distance aggregation
- GeoHash grid aggregation
- Pipeline aggregations
- Calculating the cumulative sum of usage over time
- Summary
- Chapter 5: Analyzing Log Data
- Log analysis challenges
- Logstash
- Installation and configuration
- Prerequisites
- Downloading and installing Logstash
- Installing on Windows
- Installing on Linux
- Running Logstash
- Logstash architecture
- Overview of Logstash plugins
- Installing or updating plugins
- Input plugins
- Output plugins
- Filter plugins
- Codec plugins
- Exploring plugins
- Exploring Input plugins
- File
- Beats
- JDBC
- IMAP
- Output plugins
- Elasticsearch
- CSV
- Kafka
- PagerDuty
- Codec plugins
- JSON
- Rubydebug
- Multiline
- Filter plugins
- Ingest node
- Defining a pipeline
- Ingest APIs
- Put pipeline API
- Get Pipeline API
- Delete pipeline API
- Simulate pipeline API
- Summary
- Chapter 6: Building Data Pipelines with Logstash
- Parsing and enriching logs using Logstash
- Filter plugins
- CSV filter
- Mutate filter
- Grok filter
- Date filter
- Geoip filter
- Useragent filter
- Introducing Beats
- Beats by Elastic.co
- Filebeat
- Metricbeat
- Packetbeat
- Heartbeat
- Winlogbeat
- Auditbeat
- Community Beats
- Logstash versus Beats
- Filebeat
- Downloading and installing Filebeat
- Installing on Windows
- Installing on Linux
- Architecture
- Configuring Filebeat
- Filebeat prospectors
- Filebeat global options
- Filebeat general options
- Output configuration
- Filebeat modules
- Summary
- Chapter 7: Visualizing data with Kibana
- Downloading and installing Kibana
- Installing on Windows
- Installing on Linux
- Configuring Kibana
- Data preparation
- Kibana UI
- User interaction
- Configuring the index pattern
- Discover
- Elasticsearch query string.
- Elasticsearch DSL query
- Visualize
- Kibana aggregations
- Bucket aggregations
- Metric
- Creating a visualization
- Visualization types
- Line, area, and bar charts
- Data table
- MarkDown widget
- Metric
- Goal
- Gauge
- Pie charts
- Co-ordinate maps
- Region maps
- Tag cloud
- Visualizations in action
- Response codes over time
- Top 10 URLs requested
- Bandwidth usage of top five countries over time
- Web traffic originating from different countries
- Most used user agent
- Dashboards
- Creating a dashboard
- Saving the dashboard
- Cloning the dashboard
- Sharing the dashboard
- Timelion
- Timelion UI
- Timelion expressions
- Using plugins
- Installing plugins
- Removing plugins
- Summary
- Chapter 8: Elastic X-Pack
- Installing X-Pack
- Installing X-Pack on Elasticsearch
- Installing X-Pack on Kibana
- Uninstalling X-Pack
- Configuring X-Pack
- Security
- User authentication
- User authorization
- Security in action
- New user creation
- Deleting a user
- Changing the password
- New role creation
- How to Delete/Edit a role
- Document-level security or field-level security
- X-Pack security APIs
- User management APIs
- Role management APIs
- Monitoring Elasticsearch
- Monitoring UI
- Elasticsearch metrics
- Overview tab
- Nodes tab
- The Indices tab
- Alerting
- Anatomy of a watch
- Alerting in action
- Create a new alert
- Threshold Alert
- Advanced Watch
- How to Delete/Deactivate/Edit a Watch
- Summary
- Chapter 9: Running Elastic Stack in Production
- Hosting Elastic Stack on a managed cloud
- Getting up and running on Elastic Cloud
- Using Kibana
- Overriding configuration
- Recovering from a snapshot
- Hosting Elastic Stack on your own
- Selecting hardware
- Selecting an operating system
- Configuring Elasticsearch nodes
- JVM heap size
- Disable swapping.
- File descriptors
- Thread pools and garbage collector
- Managing and monitoring Elasticsearch
- Running in Docker containers
- Special considerations while deploying to a cloud
- Choosing instance type
- Changing default ports
- do not expose ports!
- Proxy requests
- Binding HTTP to local addresses
- Installing EC2 discovery plugin
- Installing S3 repository plugin
- Setting up periodic snapshots
- Backing up and restoring
- Setting up a repository for snapshots
- Shared filesystem
- Cloud or distributed filesystems
- Taking snapshots
- Restoring a specific snapshot
- Setting up index aliases
- Understanding index aliases
- How index aliases can help
- Setting up index templates
- Defining an index template
- Creating indexes on the fly
- Modeling time series data
- Scaling the index with unpredictable volume over time
- Unit of parallelism in Elasticsearch
- The effect of the number of shards on the relevance score
- The effect of the number of shards on the accuracy of aggregations
- Changing the mapping over time
- New fields get added
- Existing fields get removed
- Automatically deleting older documents
- How index-per-timeframe solves these issues
- Scaling with index-per-timeframe
- Changing the mapping over time
- Automatically deleting older documents
- Summary
- Chapter 10: Building a Sensor Data Analytics Application
- Introduction to the application
- Understanding the sensor-generated data
- Understanding the sensor metadata
- Understanding the final stored data
- Modeling data in Elasticsearch
- Defining an index template
- Understanding the mapping
- Setting up the metadata database
- Building the Logstash data pipeline
- Accept JSON requests over the web
- Enrich the JSON with the metadata we have in the MySQL database
- The jdbc_streaming plugin
- The mutate plugin.
- Move the looked-up fields that are under lookupResult directly in JSON.
