Talend Open Studio cookbook
Getting familiar with Talend Open Studio will greatly enhance your data handling and integration capabilities. This is the perfect reference book for beginners and intermediates with a host of practical recipes that clarify even complex features. A collection of exercises covering all development as...
| Autor principal: | |
|---|---|
| Formato: | Libro electrónico |
| Idioma: | Inglés |
| Publicado: |
Birmingham :
Packt Publishing
2013.
|
| Edición: | 1st edition |
| Materias: | |
| Ver en Biblioteca Universitat Ramon Llull: | https://discovery.url.edu/permalink/34CSUC_URL/1im36ta/alma991009627867106719 |
Tabla de Contenidos:
- Intro
- Talend Open Studio Cookbook
- Table of Contents
- Talend Open Studio Cookbook
- Credits
- About the Author
- About the Reviewers
- www.PacktPub.com
- Support files, eBooks, discount offers and more
- Why Subscribe?
- Free Access for Packt account holders
- Preface
- What this book covers
- What you need for this book
- Who this book is for
- Conventions
- Reader feedback
- Customer support
- Downloading the example code
- Errata
- Piracy
- Questions
- 1. Introduction and General Principles
- Before you begin
- Installing the software
- How to do it…
- Enabling tHashInput and tHashOutput
- How to do it…
- 2. Metadata and Schemas
- Introduction
- Schema metadata
- Schemas
- Repository schemas
- Generic schemas
- Shared schemas
- Generated data sources
- Fixed schemas and columns
- Hand-cranking a built-in schema
- Getting ready
- How to do it…
- How it works…
- There's more...
- Date patterns
- Nullable elements
- Propagating schema changes
- Getting ready
- How to do it…
- How it works…
- There's more…
- Creating a generic schema from the existing metadata
- How to do it…
- How it works…
- Cutting and pasting schema information
- Getting ready
- How to do it…
- How it works…
- There's more…
- Dropping schemas to empty components
- Getting ready
- How to do it…
- How it works…
- There's more…
- Creating schemas from lists
- Getting ready
- How to do it...
- How it works…
- There's more…
- 3. Validating Data
- Introduction
- Enabling and disabling reject flows
- Getting ready
- How to do it…
- How it works…
- There's more...
- See also
- Gathering all rejects prior to killing a job
- Getting ready
- How to do it…
- How it works…
- There's more...
- See also
- Validating against the schema
- Getting ready
- How to do it…
- How it works….
- Rejecting rows using tMap
- Getting ready
- How to do it…
- How it works…
- There's more…
- Checking a column against a list of allowed values
- Getting ready
- How to do it…
- How it works…
- There's more…
- See also
- Checking a column against a lookup
- Getting ready
- How to do it…
- How it works…
- Creating validation rules for more complex requirements
- Getting ready
- How to do it…
- How it works…
- There's more…
- See also
- Creating binary error codes to store multiple test results
- Getting ready
- How to do it…
- How it works…
- There's more…
- Decrypting the error code
- 4. Mapping Data
- Introduction
- The tMap component
- Single line of code
- Batch versus real time
- Simple mapping and tMap time savers
- Getting ready
- How to do it...
- How it works...
- There's more…
- Creating tMap expressions
- Getting ready
- How to do it...
- How it works...
- There's more…
- Testing expressions
- Expression editor
- Getting around the 'one line' limitation
- See Also
- Using the ternary operator for conditional logic
- Getting ready
- How to do it...
- Single ternary expression: if-then-else
- Ternary in ternary: if-then-elsif-then-else
- How it works…
- There's more…
- Using intermediate variables in tMap
- Getting ready
- How to do it…
- How it works…
- There's more…
- Filtering input rows
- Getting ready
- How to do it...
- How it works…
- There's more…
- Splitting an input row into multiple outputs based on input conditions
- Getting ready
- How to do it...
- How it works…
- There's more…
- Joining data using tMap
- Getting ready
- How to do it...
- How it works…
- There's more…
- See Also
- Hierarchical joins using tMap
- Getting ready
- How to do it...
- How it works…
- Using reload at each row to process real-time / near real-time data
- Getting ready.
- How to do it...
- How it works…
- Loading the data into memory
- The globalMap key
- The WHERE clause
- The result
- There's more…
- 5. Using Java in Talend
- Introduction
- Performing one-off pieces of logic using tJava
- Getting ready
- How to do it…
- How it works…
- See also
- Setting the context and globalMap variables using tJava
- Getting ready
- How to do it…
- How it works…
- There's more…
- See also
- Adding complex logic into a flow using tJavaRow
- Getting ready
- How to do it…
- How it works…
- Creating pseudo components using tJavaFlex
- Getting ready
- How to do it…
- How it works…
- There's more…
- Creating custom functions using code routines
- Getting ready
- How to do it…
- How it works…
- There's more…
- See also
- Importing JAR files to allow use of external Java classes
- Getting ready
- How to do it…
- How it works…
- There's more…
- 6. Managing Context Variables
- Introduction
- Transportable code
- Context variables
- Common values in contexts
- Passing command line parameters
- Setting context variables in the code
- Database context variables
- Creating a context group
- How to do it...
- How it works...
- There's more…
- Context types
- Prompt for variable values using the tree mode
- Adding a context group to your job
- Getting ready
- How to do it...
- How it works…
- There's more…
- Adding contexts to a context group
- Getting ready
- How to do it...
- There's more…
- Using tContextLoad to load contexts
- Getting ready
- How to do it...
- How it works...
- There's more…
- Print operations
- Warnings
- Context file location
- Using implicit context loading to load contexts
- Getting ready
- How to do it...
- How it works...
- There's more…
- Turning implicit context loading on and off in a job
- Getting ready
- How to do it.
- How it works...
- Setting the context file location in the operating system
- Getting ready
- How to do it...
- How it works…
- There's more…
- Variable not present
- Implicit context load
- 7. Working with Databases
- Introduction
- Setting up a database connection
- Getting ready
- How to do it...
- How it works...
- There's more…
- Using the connection
- Always create database connections
- Connection names
- Context
- Importing the table schemas
- Getting ready
- How to do it…
- How it works...
- There's more…
- Reading from database tables
- Getting ready
- How to do it…
- Selected rows and columns
- Multiple tables and complex queries
- How it works…
- There's more…
- Efficiency versus readability
- SQL string
- SQL style
- Using context and globalMap variables in SQL queries
- Getting ready
- How to do it…
- How it works…
- There's more…
- The globalMap variables
- Developing the query
- Reloading at each row
- Printing your input query
- Getting ready
- How to do it…
- How it works…
- There's more…
- Writing to a database table
- Getting ready
- How to do it…
- How it works…
- There's more…
- Creating tables
- Update and delete keys
- Batches
- Bulk loading
- Bulk loading to temp table
- Printing your output query
- Getting ready
- How to do it…
- How it works…
- There's more…
- Managing database sessions
- Getting ready
- How to do it…
- How it works…
- Executions
- There's more…
- Multiple outputs
- Don't forget the commit
- Committing but not closing
- Passing a session to a child job
- Getting ready
- How to do it…
- How it works…
- Selecting different fields and keys for insert, update, and delete
- Getting ready
- How to do it…
- How it works…
- There's more...
- Updating
- Deleting
- Capturing individual rejects and errors
- Getting ready.
- How to do it…
- How it works…
- There's more…
- Die on error
- Efficiency
- Error management
- Database and table management
- Getting ready
- How to do it…
- How it works…
- There's more…
- Managing surrogate keys for parent and child tables
- Getting ready
- How to do it…
- How it works…
- There's more...
- Added efficiency using hashMap key table
- Ranges
- Sequences
- Auto increment keys
- The LastInsertId component
- Auto increment procedure
- Rewritable lookups using an in-process database
- Background
- Getting ready
- How to do it…
- How it works…
- In-memory components
- Initialize the data
- tMap
- Write back
- There's more…
- Memory
- See also
- 8. Managing Files
- Introduction
- Appending records to a file
- Getting ready
- How to do it...
- How it works...
- There's more…
- Concatenating files using the append method
- Reading rows using a regular expression
- Getting ready
- How to do it...
- How it works...
- There's more…
- Using temporary files
- Getting ready
- How to do it...
- How it works...
- There's more…
- See also
- Storing intermediate data in the memory using tHashMap
- Getting ready
- How to do it...
- How it works...
- There's more…
- Reading headers and trailers using tMap
- Getting ready
- How to do it...
- How it works...
- There's more…
- Reading headers and trailers with no identifiers
- Getting ready
- How to do it...
- How it works...
- Using the information in the header and trailer
- Getting ready
- How to do it...
- Validation subjob
- Use the header information subjob
- How it works...
- Validating using the trailer information
- Using the header information in the detail
- There's more…
- Adding a header and trailer to a file
- Getting ready
- How to do it...
- How it works...
- There's more…
- See also.
- Moving, copying, renaming, and deleting files and folders.
