PyTables User's Guide

Francesc Altet

Ivan Vilata

Scott Prater

Vicent Mas

Tom Hedley

Antonio Valentino

Jeffrey Whitaker


Table of Contents
I. The PyTables Core Library
1. Introduction
1.1. Main Features
1.2. The Object Tree
2. Installation
2.1. Installation from source
2.2. Binary installation (Windows)
3. Tutorials
3.1. Getting started
3.2. Browsing the object tree
3.3. Commiting data to tables and arrays
3.4. Multidimensional table cells and automatic sanity checks
3.5. Exercising the Undo/Redo feature
3.6. Using enumerated types
3.7. Dealing with nested structures in tables
3.8. Other examples in PyTables distribution
4. Library Reference
4.1. tables variables and functions
4.2. The File class
4.3. The Node class
4.4. The Group class
4.5. The Leaf class
4.6. The Table class
4.7. The Cols class
4.8. The Description class
4.9. The Column class
4.10. The Array class
4.11. The CArray class
4.12. The EArray class
4.13. The VLArray class
4.14. The UnImplemented class
4.15. The AttributeSet class
4.16. Declarative classes
4.17. Helper classes
5. Optimization tips
5.1. Informing PyTables about expected number of rows in tables
5.2. Accelerating your searches
5.3. Compression issues
5.4. Shuffling (or how to make the compression process more effective)
5.5. Using Psyco
5.6. Getting the most from the node LRU cache
5.7. Selecting an User Entry Point (UEP) in your tree
5.8. Compacting your PyTables files
II. Complementary modules
6. FileNode - simulating a filesystem with PyTables
6.1. What is FileNode?
6.2. Finding a FileNode node
6.3. FileNode - simulating files inside PyTables
6.4. Complementary notes
6.5. Current limitations
6.6. FileNode module reference
7. NetCDF - a PyTables NetCDF3 emulation API
7.1. What is NetCDF?
7.2. Using the tables.NetCDF module
7.3. tables.NetCDF module reference
7.4. Converting between true netCDF files and tables.NetCDF files
7.5. tables.NetCDF file structure
7.6. Sharing data in tables.NetCDF files over the internet with OPeNDAP
7.7. Differences between the Scientific.IO.NetCDF API and the tables.NetCDF API
III. Appendixes
A. Supported data types in PyTables
B. Using nested record arrays
B.1. Introduction
B.2. NestedRecArray methods
B.3. NestedRecord objects
C. Utilities
C.1. ptdump
C.2. ptrepack
C.3. nctoh5
D. PyTables File Format
D.1. Mandatory attributes for a File
D.2. Mandatory attributes for a Group
D.3. Mandatory attributes, storage layout and supported data types for Leaves
List of Tables
5.1. Retrieving speed and memory consumption dependency of the number of nodes in LRU cache.
A.1. Data types supported for array elements and tables columns in PyTables.
List of Figures
1.1. An HDF5 example with 2 subgroups, 2 tables and 1 array.
1.2. A PyTables object tree example.
3.1. The initial version of the data file for tutorial 1, with a view of the data objects.
3.2. The final version of the data file for tutorial 1.
3.3. General properties of the /detector/readout table.
3.4. Table hierarchy for tutorial 2.
5.1. Times for different selection modes over Int32 values. Benchmark made on a machine with Itanium (IA64) @ 900 MHz processors with SCSI disk @ 10K RPM.
5.2. Times for different selection modes over Float64 values. Benchmark made on a machine with Itanium (IA64) @ 900 MHz processors with SCSI disk @ 10K RPM.
5.3. Times for indexing a couple of columns of data type Int32 and Float64. Benchmark made on a machine with Itanium (IA64) @ 900 MHz processors with SCSI disk @ 10K RPM.
5.4. Comparison between different compression libraries.
5.5. Comparison between different compression levels of Zlib.
5.6. Writing tables with several compressors.
5.7. Selecting values in tables with several compressors. The file is not in the OS cache.
5.8. Selecting values in tables with several compressors. The file is in the OS cache.
5.9. Writing in tables with different levels of compression.
5.10. Selecting values in tables with different levels of compression. The file is in the OS cache.
5.11. Comparison between different compression libraries with and without the shuffle filter.
5.12. Writing with different compression libraries with and without the shuffle filter.
5.13. Reading with different compression libraries with the shuffle filter. The file is not in OS cache.
5.14. Reading with different compression libraries with and without the shuffle filter. The file is in OS cache.
5.15. Writing tables with/without Psyco.
5.16. Reading tables with/without Psyco.
5.17. Complete tree in file test.h5, and subtree of interest for the user.
5.18. Resulting object tree derived from the use of the rootUEP parameter.
Hierarchical datasets in Python - Release 1.3.2