Getting Started

Required configurations for use of the Python/MySQL framework:

  • A user account on the 'typhon' machine
  • Some knowledge of Unix (use of a terminal especially)
  • A functional Python distribution with the required third-party packages installed
  • Lidar Group Python scripts checked out of the SVN repository

About the Python/MySQL framework:

The MySQL database stores information and connects the different pieces of data:

  • Scans: date, type (RHI, PPI, stare), azimuth, elevation angles, etc.
  • Motions (wind fields): from which scans they were computed, using which method (OF/CC), and which set of parameters
  • Sequences (set of parameters)
  • And more (e.g., validation data from the Doppler lidar)

Thanks to the database, navigation through this complex (and huge) dataset is made much easier. Operations such as 'get the list of PPI scans between this date and that date with elevation angles greater than 4 degrees' find a simple expression, and an almost instantaneous answer.
The database can be accessed through, using the read-only account for instance.

The Python framework provides a way to conveniently access this information (no need to know SQL), and use it for analysis/visualization tasks such as displaying scans, wind fields, or computing time-series. It also controls the estimators Gale (cross-correlations/CC) and Typhoon (wavelet-based optical flow/OF).

Creating a user account on the 'typhon' machine:

This can only be done by a system administrator. The current system administrator is Chris Mauzey, so he is the one you will need to contact.

Setting up a functional Python distribution with the required third-party packages:

The Python framework that the Lidar Research Group has created makes use of several third-party packages:

  • Matplotlib provides Matlab-like plotting and visualization capabilities.
  • Mysql.connector connects to the MySQL database.
  • Numpy is the fundamental package for scientific computing.
  • PyEphem computes sunset and sunrise times.
  • Pytz deals with time zone conversions.
  • Scipy is a scientific library for advanced computations (e.g., filtering).
  • PyCuda is a CUDA wrapper for Python.
  • NetCDF4 enables handling of NetCDF files.

Currently, an ActivePython distribution is installed on typhon. It comes with many of the above packages available, though some might need to be installed on the user level (i.e., by yourself) with the 'pypm' command. To do this, log onto typhon ('ssh -p 4964 username@typhon') and type 'pypm install package-name'. To find out whether a package is already installed, log onto typhon and type 'pypm search package-name'.
The PyCuda and NetCDF4 packages are apart from pypm, so they need to be compiled and installed with the 'pip' command. To do this, log onto typhon and type 'pip install package-name'.

Checking out the Lidar Group's Python scripts from the SVN repository:

Most of the source code (Python scripts, C++, CUDA, and some IDL scripts as well) developed in the Lidar Group is stored on the Subversion (a.k.a. SVN) server located on typhon. From Wikipedia: Developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. This software keeps track of changes made to files by different authors, and therefore serves as some kind of backup or sharing system. It is extremely useful, even for personal use.
To check out the Lidar Group's Python scripts from the SVN repository, you must follow the steps below. This should create a Python directory containing several subdirectories:

  1. Create a user account on SVN@typhon (contact Chris Mauzey about this).
  2. Log onto the typhon server in a terminal using your user information: 'ssh -p 4964 username@typhon'.
  3. Create a directory where you want the Python script files to be downloaded: 'mkdir directory-name' (e.g., 'mkdir ~/Code'). Then 'cd' to that directory: 'cd directory-name' (e.g., 'cd ~/Code').
  4. Checkout (download a copy) of the Python scripts: 'svn checkout --username your-login svn://'.