Spark : big data cluster computing in production

Spark big data cluster computing in production

Production-targeted Spark guidance with real-world use cases Spark: Big Data Cluster Computing in Production goes beyond general Spark overviews to provide targeted guidance toward using lightning-fast big-data clustering in production. Written by an expert team well-known in the big data community,...

Descripción completa

Detalles Bibliográficos
Otros Autores: Ganelin, Ilya, author (author)
Formato: Libro electrónico
Idioma:Inglés
Publicado: Indianapolis, Indiana : Wiley [2016]
Edición:First edition
Colección:THEi Wiley ebooks.
Materias:
Spark (Electronic resource : Apache Software Foundation)
Big data.
Parallel processing (Electronic computers)
Electronic data processing > Distributed processing.
Data mining > Computer programs.
Ver en Biblioteca Universitat Ramon Llull:https://discovery.url.edu/permalink/34CSUC_URL/1im36ta/alma991009630150306719
Tabla de Contenidos:
  • Metrics SystemExternal Monitoring Tools; Summary; Chapter 2 Cluster Management; Background; Spark Components; Driver; Workers and Executors; Configuration; Spark Standalone; Architecture; Single-Node Setup Scenario; Multi-Node Setup; YARN; Architecture; Dynamic Resource Allocation; Scenario; Mesos; Setup; Architecture; Dynamic Resource Allocation; Basic Setup Scenario; Comparison; Summary; Chapter 3 Performance Tuning; Spark Execution Model; Partitioning; Controlling Parallelism; Partitioners; Shuffling Data; Shuffling and Data Partitioning; Operators and Shuffling
  • Shuffling Is Not That Bad After AllSerialization; Kryo Registrators; Spark Cache; Spark SQL Cache; Memory Management; Garbage Collection; Shared Variables; Broadcast Variables; Accumulators; Data Locality; Summary; Chapter 4 Security; Architecture; Security Manager; Setup Configurations; ACL; Configuration; Job Submission; Web UI; Network Security; Encryption; Event logging; Kerberos; Apache Sentry; Summary; Chapter 5 Fault Tolerance or Job Execution; Lifecycle of a Spark Job; Spark Master; Spark Driver; Spark Worker; Job Lifecycle; Job Scheduling; Scheduling within an Application
  • Scheduling with External UtilitiesFault Tolerance; Internal and External Fault Tolerance; Service Level Agreements (SLAs); Resilient Distributed Datasets (RDDs); Batch versus Streaming; Testing Strategies; Recommended Configurations; Summary; Chapter 6 Beyond Spark; Data Warehousing; Spark SQL CLI; Thrift JDBC/ODBC Server; Hive on Spark; Machine Learning; DataFrame; MLlib and ML; Mahout on Spark; Hivemall on Spark; External Frameworks; Spark Package; XGBoost; spark-jobserver; Future Works; Integration with the Parameter Server; Deep Learning; Enterprise Usage
  • Collecting User Activity Log with Spark and KafkaReal-Time Recommendation with Spark; Real-Time Categorization of Twitter Bots; Summary; Index; EULA

Ejemplares similares