# M2HATS NCAR/EOL ISFS data

## Purpose of this document

This document explains how to find, choose, read, and correctly interpret the public **NSF NCAR/EOL Integrated Surface Flux System (ISFS)** data from the 2023 M2HATS field experiment.

It is intended for:

- researchers;
- students;
- collaborators;
- programmers;
- and AI-assisted analysis workflows.

The emphasis is practical:

- Which ISFS product should I use?
- Where are the files?
- How are the variables named?
- How do the 5-minute and high-rate products differ?
- How are time coordinates represented?
- How do I avoid common errors involving fill values, sonic diagnostics, sampling rates, and variable names?
- How can MATLAB read selected files directly from the Chico State web server without first downloading a large archive?

This document is **not** a substitute for the authoritative NSF NCAR/EOL ISFS data report.

---

# 1. Source hierarchy

Use the following source hierarchy when working with M2HATS ISFS data.

## 1.1 Authoritative scientific and instrumentation reference

The primary reference for sensor locations, instrument models, sampling rates, variable definitions, processing, quality control, configuration changes, and known data issues is:

```text
https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/Data-report_M2HATS_ISFS_Flux.pdf
```

Title:

```text
M2HATS
NSF NCAR/EOL ISFS Surface Meteorology
and Flux Products Data Report
```

The report was originally released as version 1.0 and updated as document version 1.1 on 30 August 2024 to describe reprocessed products with corrected azimuth angles and Leica-based adjustments with respect to gravity.

The M2HATS ISFS **data product** is listed by NCAR/EOL as version 1.0, final.

## 1.2 Authoritative general ISFS netCDF conventions

For the general structure of ISFS netCDF files, including:

- `base_time`;
- `time`;
- placement of averaged timestamps;
- dimensions;
- sample timing;
- counts variables;

consult the NCAR/EOL ISFS netCDF documentation:

```text
https://www.eol.ucar.edu/content/isfs-netcdf-files
```

## 1.3 Chico State operational guidance

This Markdown document describes:

- the public Chico State mirror;
- recommended product choices;
- tested MATLAB remote-access methods;
- known practical pitfalls;
- examples that have been successfully used with the M2HATS files.

If a question cannot be resolved by this document and the NCAR/EOL report, inspect the actual netCDF file rather than guessing.

> **Do not invent a variable name, sensor, sampling rate, tower configuration, time convention, or quality-control rule.**

---

# 2. M2HATS ISFS in one paragraph

The M2HATS ISFS deployment included a **245-m-long horizontal array of 50 sonic-anemometer towers spaced 5 m apart**, plus two multilevel towers near the **eastern end** of the array.

The horizontal-array towers were approximately 4.5 m tall, with sonic anemometers mounted near 4.4 m AGL.

The multilevel towers provided measurements from approximately 0.5 to 28 m AGL.

The ISFS measurements include:

- three-dimensional wind;
- sonic temperature;
- thermodynamic air temperature;
- relative humidity;
- pressure;
- water vapor;
- carbon dioxide;
- radiation;
- soil temperature;
- soil moisture;
- soil heat flux;
- soil thermal properties;
- precipitation information;
- and derived turbulence and flux statistics.

The formal ISFS data period is approximately:

```text
23 July - 24 September 2023
```

For experiment-wide context, see:

```text
M2HATS_overview.md
```

---

# 3. Public ISFS data location

The public Chico State ISFS directory is:

```text
https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/
```

The main products currently visible there include:

```text
qc_geo_hr_07/
qc_geo_hr_08/
qc_geo_hr_09/

qc_geo_tiltcor_hr_07/
qc_geo_tiltcor_hr_08/
qc_geo_tiltcor_hr_09/

qc_geo_tiltcor_hr_5min/
```

The directory also contains the NCAR/EOL ISFS data report.

---

# 4. Which ISFS product should I use?

This is the most important practical choice.

## 4.1 `qc_geo_tiltcor_hr`: recommended for high-rate work

Use:

```text
qc_geo_tiltcor_hr_07/
qc_geo_tiltcor_hr_08/
qc_geo_tiltcor_hr_09/
```

for analyses that require the full native or near-native measurement rate.

The NCAR/EOL report identifies these as high-rate files with:

```text
Geographic coordinates:       Yes
Tilt corrected anemometers:   Yes
```

These are the preferred high-rate products for most new turbulence analyses.

Typical uses include:

- turbulent fluctuations;
- spectra;
- covariances;
- flux calculations;
- short-lived events;
- comparisons among sonic anemometers;
- analyses requiring 30-, 50-, or 60-Hz sonic measurements.

A representative public file is:

```text
https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/
qc_geo_tiltcor_hr_08/
isfs_m2hats_qc_geo_tiltcor_hr_20230805_00.nc
```

Each Chico high-rate file covers approximately **one hour**.

---

## 4.2 `qc_geo_tiltcor_hr_5min`: recommended for longer-period meteorology

Use:

```text
qc_geo_tiltcor_hr_5min/
```

when 5-minute averages are sufficient.

The NCAR/EOL report identifies this product as:

```text
Geographic coordinates:              Yes
Tilt adjusted with respect to gravity: Yes
Time resolution:                     5 minutes
```

This product is especially convenient for:

- full-day plots;
- multiday plots;
- diurnal cycles;
- changing weather;
- mean temperature;
- mean wind;
- pressure;
- relative humidity;
- comparing mean conditions among towers;
- examining NCAR/EOL-provided 5-minute moments and flux-related quantities.

A representative file is:

```text
https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/
qc_geo_tiltcor_hr_5min/
isfs_m2hats_qc_geo_tiltcor_5min_20230805.nc
```

These files are **daily**.

A complete day normally contains:

```text
288 records
```

because:

```text
24 hours × 12 five-minute intervals per hour = 288
```

---

## 4.3 `qc_geo_hr`: older high-rate product without tilt correction

The directories:

```text
qc_geo_hr_07/
qc_geo_hr_08/
qc_geo_hr_09/
```

contain an earlier high-rate processed product.

The NCAR/EOL report identifies these files as:

```text
Geographic coordinates:       Yes
Tilt corrected anemometers:   No
```

These files may be useful for:

- provenance;
- comparison with earlier processing;
- specialized studies that intentionally need the non-tilt-corrected components.

They should **not** normally be the first choice for new turbulence analyses when the tilt-corrected high-rate product is available.

### Important: these are not raw data

It is tempting to call `qc_geo_hr` "raw" because it precedes the later tilt correction.

That is not accurate.

NCAR's NIDAS processing system has already:

- interpreted the sensor data streams;
- applied calibrations;
- applied quality-control rules and filters;
- created scientific variables;
- written them to netCDF.

Therefore:

> `qc_geo_hr` is an **earlier processed high-rate product without the final sonic tilt correction**, not untouched raw instrument data.

---

# 5. Quick product-selection guide

| Scientific goal | Recommended product |
|---|---|
| Full-day air-temperature plot | `qc_geo_tiltcor_hr_5min` |
| Multiday weather evolution | `qc_geo_tiltcor_hr_5min` |
| Mean wind over hours or days | `qc_geo_tiltcor_hr_5min` |
| NCAR/EOL supplied 5-minute moments | `qc_geo_tiltcor_hr_5min` |
| Turbulent velocity fluctuations | `qc_geo_tiltcor_hr` |
| Spectra | `qc_geo_tiltcor_hr` |
| Covariances / turbulence fluxes | `qc_geo_tiltcor_hr` |
| Short-lived turbulence event | `qc_geo_tiltcor_hr` |
| Compare pre-/post-tilt processing | `qc_geo_hr` and `qc_geo_tiltcor_hr` |

When in doubt:

> **Use `qc_geo_tiltcor_hr_5min` for meteorological evolution and `qc_geo_tiltcor_hr` for turbulence.**

---

# 6. The horizontal array and tower naming

The 50-tower horizontal array extends from west to east.

The site names are:

```text
t0p, t01, t02, ... , t49
```

The approximate tower spacing is:

```text
5 m
```

and the total array length is:

```text
245 m
```

The array was built approximately perpendicular to the prevailing daytime southerly wind.

The NCAR/EOL report contains the surveyed coordinates and exact sensor heights.

For analyses requiring precise geometry, use the Leica-based coordinates in the report rather than assuming that all GPS values in the netCDF files have survey-level precision.

---

# 7. Multilevel tower site `t0`

Two physically separate towers near the eastern end of the horizontal array were combined conceptually into the multilevel ISFS site:

```text
t0
```

Measurements were made at approximately:

```text
0.5 m
1 m
2 m
3 m
4 m
7 m
15 m
28 m
```

The lower levels were on a short Rohn tower.

The upper levels were on a 30-m trailer tower.

## Early-campaign caution

The towers were repositioned during the first week of operations.

The NCAR/EOL report recommends against using the combined `t0` wind profile before:

```text
31 July 2023
```

when a consistent multilevel profile is required.

For work involving the early campaign, consult the report's tower-relocation section.

---

# 8. There is no universal sonic sampling rate

M2HATS used several Campbell Scientific CSAT sonic-anemometer models.

Important nominal sampling rates include:

| Sonic model | Sampling rate |
|---|---:|
| CSAT3A | 60 samples/s |
| CSAT3B | 50 samples/s |
| CSAT3 | 30 samples/s |

Other sensors use other sampling rates, for example:

| Instrument | Typical sampling rate |
|---|---:|
| Sensirion SHT85 temperature/RH | 1 sample/s |
| Paroscientific nanobarometer | 20 samples/s |

Therefore:

> **Never assume all M2HATS sonic data are 60 Hz.**

Before constructing:

- spectra;
- filters;
- averaging windows;
- high-rate time axes;
- sample counts;

determine the actual instrument and variable dimensions.

Use:

1. Appendix A of the ISFS report for sonic model/site information;
2. Appendix B for the high-rate variable dictionary and sampling rates;
3. the actual netCDF file dimensions.

---

# 9. Variable naming convention

The general M2HATS ISFS variable naming convention is:

```text
<variable>_<height>_<site>
```

Examples include:

```text
T_4m_t23
RH_4m_t23
P_4m_t35
spd_15m_t0
u_4m_t23
v_4m_t23
w_4m_t23
tc_4m_t23
```

The NCAR report may display a `short_name` using periods, for example:

```text
T.4m.t23
u.4m.t23
```

but the actual MATLAB/netCDF variable names use underscores:

```text
T_4m_t23
u_4m_t23
```

Variable names encode physical meaning.

Do not treat similar-looking variables as interchangeable.

---

# 10. Critical distinction: `T_*` versus `tc_*`

This distinction is essential.

## 10.1 `T_*`: thermodynamic air temperature

A variable such as:

```text
T_4m_t23
```

is air temperature measured by the **Sensirion SHT85 hygro-thermometer**.

Use `T_*` for:

- diurnal cycles;
- ordinary meteorological air temperature;
- slowly changing temperature;
- multihour or multiday plots;
- comparison of mean near-surface temperature.

The SHT85 measurement is typically much slower than the sonic measurement.

---

## 10.2 `tc_*`: sonic virtual temperature

A variable such as:

```text
tc_4m_t23
```

is the temperature quantity inferred from the speed of sound measured by the sonic anemometer.

It accompanies the high-rate sonic velocity variables.

Use `tc_*` for:

- turbulent temperature fluctuations;
- sonic-temperature covariance;
- high-frequency turbulence analysis;
- flux work where sonic temperature is the intended quantity.

---

## 10.3 Do not substitute one for the other

Do **not** silently replace:

```text
T_*
```

with:

```text
tc_*
```

or vice versa.

They come from different instruments, have different sampling characteristics, and represent different physical quantities.

---

# 11. High-rate and 5-minute files do not contain identical variable sets

Do not assume that a variable available in the 5-minute product is also stored explicitly in the high-rate product.

For example, the 5-minute product may contain:

```text
spd_4m_t23
```

as horizontal wind speed.

But the high-rate t23 sonic data contain the velocity components:

```text
u_4m_t23
v_4m_t23
w_4m_t23
tc_4m_t23
```

and may not contain a stored high-rate `spd_4m_t23`.

For high-rate horizontal wind speed, calculate:

```matlab
spd = sqrt(u.^2 + v.^2);
```

after quality control.

> Before writing high-rate analysis code, verify that the exact variable exists.

---

# 12. 5-minute moments and flux-related products

The 5-minute files contain not only ordinary means but also processed moments and flux-related quantities.

For tower 23 at 4 m, verified examples include:

```text
T_4m_t23
P_4m_t23
spd_4m_t23
dir_4m_t23
u_u__4m_t23
v_v__4m_t23
w_w__4m_t23
w_tc__4m_t23
```

The variables:

```text
u_u__4m_t23
v_v__4m_t23
w_w__4m_t23
```

are stored velocity second moments / variances.

A 5-minute turbulent kinetic energy estimate can therefore be formed as:

```matlab
TKE = 0.5 .* (uu + vv + ww);
```

Do not assume a standalone variable named:

```text
TKE
```

exists.

## Important averaging caution

The NCAR/EOL report warns that a 5-minute averaging period can be too short to capture all turbulence scales contributing to total flux, especially during unstable daytime conditions.

Therefore distinguish among:

1. plotting an NCAR/EOL-supplied 5-minute quantity;
2. combining 5-minute moments into a longer averaging period using the documented method;
3. independently calculating turbulence statistics from high-rate measurements.

These are not the same operation.

---

# 13. Time variables

ISFS files normally contain:

```text
base_time
time
```

A common MATLAB pattern for variables with one value per `time` record is:

```matlab
base_id = netcdf.inqVarID(ncid,'base_time');
time_id = netcdf.inqVarID(ncid,'time');

base_time = double(netcdf.getVar(ncid,base_id));
time_sec  = double(netcdf.getVar(ncid,time_id));

t = datetime(base_time + time_sec, ...
    'ConvertFrom','posixtime', ...
    'TimeZone','UTC');
```

M2HATS occurred during Pacific Daylight Time.

For local time:

```matlab
t.TimeZone = 'America/Los_Angeles';
```

Keep data in UTC when UTC is scientifically or operationally preferable.

---

# 14. Five-minute timestamps are interval centers

For time-averaged ISFS products, the stored timestamp represents the **middle of the averaging interval**.

For a 5-minute average:

```text
300 s averaging interval
```

the timestamps occur approximately:

```text
00:02:30
00:07:30
00:12:30
...
```

Thus:

```text
14:02:30
```

represents approximately:

```text
14:00:00 - 14:05:00
```

and:

```text
14:27:30
```

represents approximately:

```text
14:25:00 - 14:30:00
```

Do not interpret a stored 5-minute timestamp as the beginning or end of the averaging interval.

This matters when comparing 5-minute products with high-rate data.

---

# 15. High-rate time structure

A high-rate sonic variable does not necessarily have one independent netCDF `time` value for every sonic sample.

A verified t23 hourly file has a structure like:

```text
time       -> 3600 records
w_4m_t23   -> 60 x 3600
```

The first dimension of `w_4m_t23` contains the sub-second measurements associated with each one-second record.

Therefore:

> **Do not infer the sonic sampling rate from `diff(time)`.**

`diff(time)` reflects the record interval, not necessarily the sub-sample interval.

---

# 16. Exact high-rate sub-second timing

For a variable with:

```text
sample
```

sub-samples associated with each time record and a sample rate:

```text
R samples/s
```

the ISFS convention places samples evenly around the record time.

The timing relationship is:

```text
t(i,j) = base_time + time(i)
         - dT/2
         + (dT/sample)*(1/2 + j)

dT = sample/R

j = 0, 1, ..., sample-1
```

For the common case where:

```text
sample = R
```

then:

```text
dT = 1 s
```

For a verified 60-Hz variable:

```matlab
Fs = size(w,1);
dT = 1;

t_record = datetime(base_time + time_sec, ...
    'ConvertFrom','posixtime', ...
    'TimeZone','UTC');

subsec = -dT/2 + ((0:Fs-1)' + 0.5)*(dT/Fs);

t_high = t_record(:).' + seconds(subsec);
```

The resulting:

```text
t_high
```

has the same shape as the high-rate sonic variable.

Plot with:

```matlab
plot(t_high(:),w(:))
```

Do not use:

```matlab
t_record(:).' + seconds((0:Fs-1)'/Fs)
```

when exact ISFS timing matters.

That approximation incorrectly begins the sub-samples at the nominal record time rather than centering them around it.

---

# 17. Fill values

Low-level MATLAB calls using:

```matlab
netcdf.getVar
```

can return the raw netCDF `_FillValue`.

For many M2HATS variables, a fill value may be near:

```text
1.e37
```

If this value is used in arithmetic, it can produce enormous and obviously nonphysical results.

Always read the actual `_FillValue` attribute.

Example:

```matlab
u_id = netcdf.inqVarID(ncid,'u_4m_t23');
u = double(netcdf.getVar(ncid,u_id));

u_fill = double(netcdf.getAtt(ncid,u_id,'_FillValue'));

u(u == u_fill) = NaN;
```

Do not hard-code `1.e37` if the file provides the actual attribute.

---

# 18. Sonic diagnostic flags

For high-rate sonic data, use the associated logical diagnostic variable.

For example:

```text
ldiag_4m_t23
```

A robust pattern is:

```matlab
ldiag_id = netcdf.inqVarID(ncid,'ldiag_4m_t23');
ldiag = double(netcdf.getVar(ncid,ldiag_id));

bad = (ldiag ~= 0);

u(bad)  = NaN;
v(bad)  = NaN;
w(bad)  = NaN;
tc(bad) = NaN;
```

Apply fill-value screening as well.

Then use NaN-aware calculations such as:

```matlab
mean(x,'omitnan')
```

Do not calculate wind speed, spectra, covariances, or averages before screening bad and missing sonic samples.

---

# 19. MATLAB direct access from the Chico server

A major advantage of the Chico public mirror is that MATLAB can read many netCDF files **directly over HTTPS**.

For MATLAB remote netCDF access, append:

```text
#mode=bytes
```

to the URL.

Example:

```matlab
url = ['https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/' ...
       'qc_geo_tiltcor_hr_08/' ...
       'isfs_m2hats_qc_geo_tiltcor_hr_20230805_00.nc#mode=bytes'];
```

This allows MATLAB to use HTTP byte-range access.

The entire file does not need to be copied manually to the local computer before analysis.

---

# 20. Chico files were optimized for remote access

The original classic-netCDF files used an:

```text
UNLIMITED time dimension
```

which caused extremely inefficient remote access over HTTP.

The Chico State copies were repacked server-side with:

```bash
nccopy -u original.nc optimized.nc
```

This converted the completed unlimited time dimension into a fixed-size dimension while retaining classic netCDF format.

Conceptually:

```text
Before:
time = UNLIMITED ; // (3600 currently)

After:
time = 3600 ;
```

This changes the storage layout to make remote byte-range access much more efficient.

It is not intended to change the scientific data values.

Students and users should **not** run `nccopy` themselves on the public files.

The Chico copies have already been prepared for direct remote access.

---

# 21. Open a remote file once

Remote file opening can be much more expensive than reading an additional variable from an already-open file.

Therefore, when several variables are needed from one file:

> **Open once, read everything needed, close once.**

Preferred pattern:

```matlab
ncid = netcdf.open(url,'NOWRITE');

u_id  = netcdf.inqVarID(ncid,'u_4m_t23');
v_id  = netcdf.inqVarID(ncid,'v_4m_t23');
w_id  = netcdf.inqVarID(ncid,'w_4m_t23');
tc_id = netcdf.inqVarID(ncid,'tc_4m_t23');

u  = double(netcdf.getVar(ncid,u_id));
v  = double(netcdf.getVar(ncid,v_id));
w  = double(netcdf.getVar(ncid,w_id));
tc = double(netcdf.getVar(ncid,tc_id));

netcdf.close(ncid);
```

Repeated independent calls to:

```matlab
ncread()
```

may repeatedly incur remote file-open and header costs.

They can work, but they are not normally the most efficient pattern for multiple variables from the same remote file.

---

# 22. Known-good MATLAB example: one day of 5-minute air temperature

This example reads the 4-m SHT85 air-temperature measurement at tower 23 for 5 August 2023.

```matlab
clear
close all
clc

url = ['https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/' ...
       'qc_geo_tiltcor_hr_5min/' ...
       'isfs_m2hats_qc_geo_tiltcor_5min_20230805.nc#mode=bytes'];

ncid = netcdf.open(url,'NOWRITE');

base_id = netcdf.inqVarID(ncid,'base_time');
time_id = netcdf.inqVarID(ncid,'time');
T_id    = netcdf.inqVarID(ncid,'T_4m_t23');

base_time = double(netcdf.getVar(ncid,base_id));
time_sec  = double(netcdf.getVar(ncid,time_id));
T         = double(netcdf.getVar(ncid,T_id));

% Replace the file's fill value with NaN.
T_fill = double(netcdf.getAtt(ncid,T_id,'_FillValue'));
T(T == T_fill) = NaN;

netcdf.close(ncid);

t = datetime(base_time + time_sec, ...
    'ConvertFrom','posixtime', ...
    'TimeZone','UTC');

t.TimeZone = 'America/Los_Angeles';

figure
plot(t,T,'LineWidth',1.5)
grid on

xlabel('Local Time')
ylabel('Air Temperature (^oC)')
title('M2HATS: 4-m Air Temperature at Tower 23')
```

A complete daily file should normally produce:

```text
288 five-minute values
```

Remember that each timestamp represents the **center** of its 5-minute averaging period.

---

# 23. Known-good MATLAB example: one hour of high-rate sonic data

This example reads one hour of tilt-corrected sonic data from tower 23.

Tower 23 is a verified 60-Hz example.

Do not assume another tower has the same rate.

```matlab
clear
close all
clc

url = ['https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/' ...
       'qc_geo_tiltcor_hr_08/' ...
       'isfs_m2hats_qc_geo_tiltcor_hr_20230805_00.nc#mode=bytes'];

ncid = netcdf.open(url,'NOWRITE');

% Time variables.
base_id = netcdf.inqVarID(ncid,'base_time');
time_id = netcdf.inqVarID(ncid,'time');

base_time = double(netcdf.getVar(ncid,base_id));
time_sec  = double(netcdf.getVar(ncid,time_id));

% Sonic variables.
u_id     = netcdf.inqVarID(ncid,'u_4m_t23');
v_id     = netcdf.inqVarID(ncid,'v_4m_t23');
w_id     = netcdf.inqVarID(ncid,'w_4m_t23');
tc_id    = netcdf.inqVarID(ncid,'tc_4m_t23');
ldiag_id = netcdf.inqVarID(ncid,'ldiag_4m_t23');

u     = double(netcdf.getVar(ncid,u_id));
v     = double(netcdf.getVar(ncid,v_id));
w     = double(netcdf.getVar(ncid,w_id));
tc    = double(netcdf.getVar(ncid,tc_id));
ldiag = double(netcdf.getVar(ncid,ldiag_id));

% Read and apply each variable's fill value.
u_fill  = double(netcdf.getAtt(ncid,u_id,'_FillValue'));
v_fill  = double(netcdf.getAtt(ncid,v_id,'_FillValue'));
w_fill  = double(netcdf.getAtt(ncid,w_id,'_FillValue'));
tc_fill = double(netcdf.getAtt(ncid,tc_id,'_FillValue'));

u(u == u_fill)   = NaN;
v(v == v_fill)   = NaN;
w(w == w_fill)   = NaN;
tc(tc == tc_fill) = NaN;

netcdf.close(ncid);

% Apply sonic diagnostic screening.
bad = (ldiag ~= 0);

u(bad)  = NaN;
v(bad)  = NaN;
w(bad)  = NaN;
tc(bad) = NaN;

% Verify the sample dimension.
Fs = size(w,1);

% This particular t23 example is known to be 60 Hz.
fprintf('Samples per record: %d\n',Fs);

% Construct record-center times.
t_record = datetime(base_time + time_sec, ...
    'ConvertFrom','posixtime', ...
    'TimeZone','UTC');

% Exact ISFS centered sub-second convention.
dT = 1.0;

subsec = -dT/2 + ((0:Fs-1)' + 0.5)*(dT/Fs);

t_high = t_record(:).' + seconds(subsec);

figure
plot(t_high(:),w(:))
grid on

xlabel('Time (UTC)')
ylabel('Vertical velocity, w (m s^{-1})')
title('M2HATS: High-rate vertical velocity at tower 23')
```

For this verified tower-23 example:

```text
u, v, w, tc -> 60 x 3600
```

The same dimensional assumption must **not** be transferred automatically to another tower.

---

# 24. Deriving high-rate horizontal wind speed

A high-rate `spd_*` variable may not be stored.

After fill-value and diagnostic screening:

```matlab
spd = sqrt(u.^2 + v.^2);
```

This gives the instantaneous horizontal wind-speed magnitude from the tilt-corrected sonic components.

Do not calculate:

```matlab
sqrt(u.^2 + v.^2)
```

before invalid samples have been converted to `NaN`.

---

# 25. Comparing high-rate and 5-minute data

When comparing high-rate measurements with stored 5-minute means, variances, covariances, or flux-related products:

1. Treat each 5-minute timestamp as the center of a 300-s interval.
2. Match high-rate samples to approximately:

```text
t5 - 150 s
through
t5 + 150 s
```

3. Apply appropriate fill-value and QC screening first.
4. Do not infer a physical discrepancy until timing and QC are aligned.
5. If reproducing an NCAR/EOL moment from high-rate data, use the same averaging window and processing conventions as closely as possible.

---

# 26. Sensor availability differs by tower

Do not assume that every measurement exists at every tower.

Examples:

- every horizontal-array tower has a sonic;
- additional temperature/RH, pressure, and gas measurements occur only at selected locations;
- soil sensors occur at selected sites;
- radiometers are located near selected array positions;
- instrument models differ among sonic sites.

For serious tower-to-tower comparisons:

1. consult the ISFS report;
2. use Appendix A for sonic model/site information;
3. use Appendix B for high-rate variables and sampling rates;
4. inspect the actual netCDF variables.

---

# 27. Data-quality and configuration changes matter

The campaign included:

- instrument replacements;
- calibration issues;
- tower relocation early in the experiment;
- sonic-model differences;
- geometric corrections;
- tilt corrections;
- occasional missing or flagged samples;
- weather-related interruptions.

The NCAR/EOL report contains detailed data-quality sections by instrument.

A variable being present in a netCDF file does **not** guarantee that every sample is scientifically usable.

---

# 28. Useful file-inspection commands in MATLAB

To inspect a remote file:

```matlab
info = ncinfo(url);
```

To see the variable names:

```matlab
{info.Variables.Name}'
```

To inspect a specific variable:

```matlab
vinfo = ncinfo(url,'u_4m_t23');
```

To test whether a variable exists using the low-level interface:

```matlab
ncid = netcdf.open(url,'NOWRITE');

try
    varid = netcdf.inqVarID(ncid,'u_4m_t23');
    disp('Variable exists')
catch
    disp('Variable not found')
end

netcdf.close(ncid);
```

When an expected variable is absent, do not guess a replacement name.

Consult the report or inspect the file.

---

# 29. Recommended workflow for a new analysis

A reliable workflow is:

```text
1. Define the scientific question.
        |
        v
2. Decide whether 5-minute or high-rate data are needed.
        |
        v
3. Identify the tower/site and sensor.
        |
        v
4. Verify the variable in the report or netCDF file.
        |
        v
5. Verify the sampling rate / dimensions.
        |
        v
6. Read the file directly from the Chico server.
        |
        v
7. Replace fill values with NaN.
        |
        v
8. Apply diagnostic/QC screening.
        |
        v
9. Construct the correct time axis.
        |
        v
10. Perform the scientific calculation.
        |
        v
11. Check that the result is physically reasonable.
```

A program that runs without an error is not necessarily scientifically correct.

---

# 30. Common mistakes

## Mistake 1: treating `qc_geo_hr` as raw data

It is not raw.

It is already a calibrated, quality-controlled, processed netCDF product, but its sonic components have not received the later tilt correction.

---

## Mistake 2: assuming every sonic is 60 Hz

M2HATS used 30-, 50-, and 60-Hz sonic configurations.

Check the instrument and variable dimensions.

---

## Mistake 3: using `diff(time)` to infer sonic sampling rate

The high-rate file may have one `time` record per second while the sonic variable contains many sub-samples per record.

---

## Mistake 4: confusing `T_*` and `tc_*`

`T_*` and `tc_*` come from different measurement systems and serve different scientific purposes.

---

## Mistake 5: assuming a 5-minute variable also exists in the high-rate product

The variable sets differ.

Verify the exact high-rate variable.

---

## Mistake 6: ignoring `_FillValue`

Fill values can be around `1.e37`.

They must be converted to `NaN` before calculations.

---

## Mistake 7: ignoring `ldiag_*`

Bad sonic samples should not be treated as valid turbulence measurements.

---

## Mistake 8: treating a 5-minute timestamp as the interval start

It represents the interval center.

---

## Mistake 9: using the old non-centered high-rate timing approximation

Use the official ISFS centered sample-timing convention.

---

## Mistake 10: downloading hundreds of megabytes unnecessarily

For many tasks, MATLAB can read selected remote files directly over HTTPS from the Chico server.

---

# 31. AI-assisted use

These data work well with AI-assisted programming when the AI is given enough context.

For best results, provide:

```text
M2HATS_overview.md
M2HATS_data_locations.md
M2HATS_ISFS_data.md
```

and, for questions involving sensor availability, detailed processing, or data quality, also provide:

```text
Data-report_M2HATS_ISFS_Flux.pdf
```

An AI should be instructed to:

- use the tilt-corrected products by default;
- choose 5-minute versus high-rate data based on the scientific question;
- inspect rather than invent variables;
- distinguish `T_*` from `tc_*`;
- verify sonic sampling rate;
- apply fill-value and diagnostic screening;
- use the correct ISFS time convention;
- avoid unnecessary full-file downloads.

---

# 32. Data citation

Researchers using the NCAR/EOL ISFS data should use the dataset citation given in the NCAR report:

> NSF NCAR/EOL ISFS Team. 2024. *M2HATS: ISFS Surface Meteorology and Flux Products*. Version 1.0. UCAR/NCAR - Earth Observing Laboratory.

DOI:

```text
https://doi.org/10.26023/HW9Z-MF0D-NX04
```

Users of EOL data should also follow the acknowledgment guidance in the NCAR/EOL report.

---

# 33. Related M2HATS documentation

Experiment overview:

```text
M2HATS_overview.md
```

Public data map:

```text
M2HATS_data_locations.md
```

REAL documentation:

```text
REAL_instrument_description.md
REAL_bscan_data_file_format.md
REAL_bscan_signal_processing.md
```

Authoritative ISFS report:

```text
https://lidar.csuchico.edu/M2HATS-data/NCAR/ISFS/Data-report_M2HATS_ISFS_Flux.pdf
```

General NCAR/EOL ISFS netCDF conventions:

```text
https://www.eol.ucar.edu/content/isfs-netcdf-files
```

---

# 34. Compact summary

> For most M2HATS ISFS work, use `qc_geo_tiltcor_hr_5min` for full-day, multiday, and mean-meteorology analyses, and use `qc_geo_tiltcor_hr` when native high-rate turbulence measurements are required. The older `qc_geo_hr` product is geographically referenced and quality controlled but does not include the later sonic tilt correction, so it should not be considered raw data. Variable availability, sonic model, and sampling rate vary by site. Never assume every sonic is 60 Hz, never confuse `T_*` with `tc_*`, and never infer high-rate variable availability from the 5-minute product. When reading files with low-level MATLAB netCDF functions, replace `_FillValue` samples with `NaN`, apply sonic diagnostic flags, and use the official centered ISFS time convention. The Chico State public mirror is optimized so that MATLAB can read selected netCDF files directly over HTTPS without first downloading the full data archive.
