OpenDX — routines to read and write simple OpenDX files

The OpenDX format for multi-dimensional grid data. OpenDX is a free visualization software, see http://www.opendx.org.

Note

This module only implements a primitive subset, sufficient to represent n-dimensional regular grids.

The OpenDX scalar file format is specified in Appendix B.2 Data Explorer Native Files [1].

If you want to build a dx object from your data you can either use the convenient Grid class from the top level module (gridData.Grid) or see the lower-level methods described below.

Reading and writing OpenDX files

If you have OpenDX files from other software and you just want to read it into a Python array then you do not really need to use the interface in gridData.OpenDX: just use Grid and load the file:

from gridData import Grid
g = Grid("data.dx")

This should work for files produced by common visualization programs (VMD, PyMOL, Chimera). The documentation for gridData tells you more about what to do with the Grid object.

If you want to write an OpenDX file then you just use the gridData.core.Grid.export() method with file_format=”dx” (or just use a filename with extension “.dx”):

g.export("data.dx")

However, some visualization programs do not implement full OpenDX specifications and only read very specific, “OpenDX-like” files. gridData.OpenDX tries to be compatible with these formats. However, sometimes additional help is needed to write an OpenDX file that can be read by a specific software, as described below:

Known issues for writing OpenDX files

• APBS require the delta to be written to the seventh significant figure. The delta is now written to reflect this increase in precision.

  Changed in version 0.6.0.

• PyMOL requires OpenDX files with the type specification “double” in the class array section (see issue #35). By default (since release 0.4.0), the type is set to the one that most closely approximates the dtype of the numpy array Grid.grid, which holds all data. This is often numpy.float64, which will create an OpenDX type “double”, which PyMOL will read.

  However, if you want to force a specific OpenDX type (such as “float” or “double”, see gridData.OpenDX.array.dx_types for available values) then you can use the type keyword argument:

  g.export("for_pymol.dx", type="double")
  

  If you always want to be able to read OpenDX files with PyMOL, it is suggested to always export with type="double".

  Added in version 0.4.0.

Building a dx object from a numpy array A

If you have a numpy array A that represents a density in cartesian space then you can construct a dx object (named a field in OpenDX parlance) if you provide some additional information that fixes the coordinate system in space and defines the units along the axes.

The following data are required:

grid

numpy nD array (typically a nD histogram)

grid.shape

the shape of the array

origin

the cartesian coordinates of the center of the (0,0,..,0) grid cell

delta

\(n \times n\) array with the length of a grid cell along each axis; for regular rectangular grids the off-diagonal elements are 0 and the diagonal ones correspond to the ‘bin width’ of the histogram, eg delta[0,0] = 1.0 (Angstrom)

The DX data type (“type” in the DX file) is determined from the numpy.dtype of the numpy.ndarray that is provided as the grid (or with the type keyword argument to gridData.OpenDX.array).

For example, to build a field:

dx = OpenDX.field('density')
dx.add('positions', OpenDX.gridpositions(1, grid.shape, origin, delta))
dx.add('connections', OpenDX.gridconnections(2, grid.shape))
dx.add('data', OpenDX.array(3, grid))

or all with the constructor:

dx = OpenDX.field('density', components=dict(
          positions=OpenDX.gridpositions(1,grid.shape, d.origin, d.delta),
          connections=OpenDX.gridconnections(2, grid.shape),
          data=OpenDX.array(3, grid)))

Building a dx object from a dx file

One can also read data from an existing dx file:

dx = OpenDX.field(0)
dx.read('file.dx')

Only simple arrays are read and initially stored as a 1-d numpy.ndarray in the dx.components[‘data’].array with the numpy.dtype determined by the DX type in the file.

The dx field object has a method histogramdd() that produces output identical to the numpy.histogramdd() function by taking the stored dimension and deltas into account. In this way, one can store nD histograms in a portable and universal manner:

histogram, edges = dx.histogramdd()

[1]

The original link to the OpenDX file format specs http://opendx.sdsc.edu/docs/html/pages/usrgu068.htm#HDREDF is dead so I am linking to an archived copy at the Internet Archive , B.2 Data Explorer Native Files.

Classes and functions

class gridData.OpenDX.DXInitObject(classtype, classid)

Storage class that holds data to initialize one of the ‘real’ classes such as OpenDX.array, OpenDX.gridconnections, …

All variables are stored in args which will be turned into the arguments for the DX class.

initialize()

Initialize the corresponding DXclass from the data.

class = DXInitObject.initialize()

exception gridData.OpenDX.DXParseError

general exception for parsing errors in DX files

class gridData.OpenDX.DXParser(filename)

Brain-dead baroque implementation to read a simple (VMD) dx file.

Requires a OpenDX.field instance.

1. scan for ‘object’ lines: ‘object’ id ‘class’ class [data] [data …]

2. parse data according to class

3. construct dx field from classes

Setup a parser for a simple DX file (from VMD)

>>> DXfield_object = OpenDX.field(id)
>>> p = DXparser('bulk.dx')
>>> p.parse(DXfield_object)

The field object will be completely rewritten (including the id if one is found in the input file. The input files component layout is currently ignored.

Note that quotes are removed from quoted strings.

apply_parser()

Apply the current parser to the token stream.

parse(DXfield)

Parse the dx file and construct a DX field object with component classes.

A field instance DXfield must be provided to be filled by the parser:

DXfield_object = OpenDX.field(*args)
parse(DXfield_object)

A tokenizer turns the dx file into a stream of tokens. A hierarchy of parsers examines the stream. The level-0 parser (‘general’) distinguishes comments and objects (level-1). The object parser calls level-3 parsers depending on the object found. The basic idea is that of a ‘state machine’. There is one parser active at any time. The main loop is the general parser.

• Constructing the dx objects with classtype and classid is not implemented yet.

• Unknown tokens raise an exception.

set_parser(parsername)

Set parsername as the current parser.

use_parser(parsername)

Set parsername as the current parser and apply it.

exception gridData.OpenDX.DXParserNoTokens

raised when the token buffer is exhausted

class gridData.OpenDX.DXclass(classid)

‘class’ object as defined by OpenDX

id is the object number

ndformat(s)

Returns a string with as many repetitions of s as self has dimensions (derived from shape)

write(stream, optstring='', quote=False)

write the ‘object’ line; additional args are packed in string

class gridData.OpenDX.array(classid, array=None, type=None, typequote='"', **kwargs)

OpenDX array class.

See Array Objects for details.

Parameters:

• classid (int)

• array (array_like)

• type (str (optional)) –

  Set the DX type in the output file and cast array to the closest numpy dtype. type must be one of the allowed types in DX files as defined under Array Objects. The default None tries to set the type from the numpy.dtype of array.

  Added in version 0.4.0.

Raises:

ValueError – if array is not provided; or if type is not of the correct DX type

dx_types = {'byte': 'uint8', 'double': 'float64', 'float': 'float32', 'int': 'int32', 'short': 'int16', 'signed byte': 'int8', 'signed int': 'int32', 'signed short': 'int16', 'unsigned byte': 'uint8', 'unsigned int': 'uint32', 'unsigned short': 'uint16'}

conversion from OpenDX type to closest numpy.dtype (round-tripping is not guaranteed to produce identical types); not all types are supported (e.g., strings and conversion to int64 are missing)

np_types = {'float16': 'float', 'float32': 'float', 'float64': 'double', 'int16': 'short', 'int32': 'int', 'int64': 'int', 'int8': 'signed byte', 'uint16': 'unsigned short', 'uint32': 'unsigned int', 'uint64': 'unsigned int', 'uint8': 'byte'}

conversion from numpy.dtype.name to closest OpenDX array type (round-tripping is not guaranteed to produce identical types); not all types are supported (e.g., strings are missing)

write(stream)

Write the class array section.

Parameters:

stream (stream)

Raises:

ValueError – If the dxtype is not a valid type, ValueError is raised.

class gridData.OpenDX.field(classid='0', components=None, comments=None)

OpenDX container class

The field is the top-level object and represents the whole OpenDX file. It contains a number of other objects.

Instantiate a DX object from this class and add subclasses with add().

OpenDX object, which is build from a list of components.

Parameters:

• id (str) – arbitrary string

• components (dict) –

  dictionary of DXclass instances (no sanity check on the individual ids!) which correspond to

  • positions

  • connections

  • data

• comments (list) – list of strings; each string becomes a comment line prefixed with ‘#’. Avoid newlines.

A field must have at least the components ‘positions’, ‘connections’, and ‘data’. Those components are associated with objects belonging to the field. When writing a dx file from the field, only the required objects are dumped to the file.

(For a more general class that can use field: Because there could be more objects than components, we keep a separate object list. When dumping the dx file, first all objects are written and then the field object describes its components. Objects are referenced by their unique id.)

Note

uniqueness of the id is not checked.

Example

Create a new dx object:

dx = OpenDX.field('density',[gridpoints,gridconnections,array])

add(component, DXobj)

add a component to the field

add_comment(comment)

add comments

histogramdd()

Return array data as (edges,grid), i.e. a numpy nD histogram.

read(stream)

Read DX field from file.

dx = OpenDX.field.read(dxfile)

The classid is discarded and replaced with the one from the file.

sorted_components()

iterator that returns (component,object) in id order

write(filename)

Write the complete dx object to the file.

This is the simple OpenDX format which includes the data into the header via the ‘object array … data follows’ statement.

Only simple regular arrays are supported.

The format should be compatible with VMD’s dx reader plugin.

class gridData.OpenDX.gridconnections(classid, shape=None, **kwargs)

OpenDX gridconnections class

id is the object number

write(stream)

write the ‘object’ line; additional args are packed in string

class gridData.OpenDX.gridpositions(classid, shape=None, origin=None, delta=None, **kwargs)

OpenDX gridpositions class.

shape D-tuplet describing size in each dimension origin coordinates of the centre of the grid cell with index 0,0,…,0 delta DxD array describing the deltas

id is the object number

edges()

Edges of the grid cells, origin at centre of 0,0,..,0 grid cell.

Only works for regular, orthonormal grids.

write(stream)

write the ‘object’ line; additional args are packed in string