Technical Notes

Technical Notes#

This chapter provides a technical back-story for this oceanography Jupyter Book. Both this chapter and the chapter on Documentation enable us to provide useful explanatory material while reducing clutter in the science narrative.

Shallow profiler section#

Study site locations#

The regional cabled array (RCA) maintains three shallow profiler sites in the northeast Pacific Ocean.

Site name             Abbreviation     Latitude          Longitude
------------------    ------------     --------          ---------
Oregon Offshore         OOF            44.37415          -124.95648
Oregon Slope Base       OSB            44.52897          -125.38966 
Axial Base              AXB            45.83049          -129.75326

  • needed: map image

Data dictionary d{}, comparing sensors using ChartTwoSensors()#

The epipelargosy chapter begins by building a data dictionary d{}

  • keys are sensor names (‘temp’, ‘ph’ etcetera)

    • All sensor names are given in the data chapter

  • d[key] is a 5-element tuple

    • First two elements are XArray DataFrames

      • d[key][0] is sensor data as a function of time

      • d[key][1] is depth as a function of (the same) time

    • Elements [2] and [3] are the expected numerical range of the sensor data

    • Element [4] is a string: Default plot color for this sensor

Note that if one treats a profile as an ‘instantaneous’ event at some characteristic time: The sensor value can be recast in terms of the depth dimension/coordinate.

Data dictionary values
    [0] DataArray: sensor data values, dimension = 'time'  
    [1] DataArray: sensor depths z,    dimension = 'time'    
    [2] low-end expected data range                         
    [3] high-end expected data range                      
    [4] color for this sensor                             

print(d['ph'][0], d['ph'][1])
for a in list(d['ph'][0].attrs.keys()): del d['ph'][0].attrs[a]
for a in list(d['ph'][1].attrs.keys()): del d['ph'][1].attrs[a]
print(d['ph'][0], d['ph'][1])

Profile data from two sensors can be compared on a single chart using ChartTwoSensors() found in the charts.py module.

  • There are three sites that support shallow profilers

  • Each site has a platform tethered at 200 meters depth

  • Each platform has a winch that (nine times daily) allows the profiler to rise to near the surface and descend again

  • As a result we have sensor profiles from the instruments affixed to the profiler

  • We have pressure / depth data as well so we can produce profile charts

    • These have depth as the vertical access and sensor reading on the horizontal axis

  • We also have a set of instruments attached to the platform

    • This maintains a fairly constant depth although it is subject to two types of depth-variability

      • First type: Tides (typically a couple of meters)

      • Second type: Strong currents (e.g. during a storm) drag the platform laterally

        • Because it is tethered: The net effect is the platform is pushed deeper

        • This can also affect the profiler

To further reduce clutter there are Python modules available as well: shallowprofiler.py and so forth.

From the imported shallowprofiler.py module we have available sensors: A list of lists. Each entry is [sensor_name, instrument_name] so for example sensors[4] is ['temperature', 'ctd']. To see the entirety: sensors.

The data volume across many sensors get fairly large. There are multiple ways to approach not bogging down the Jupyter/IPython environment:

  • Work at full time resolution in a narrow time window (e.g. one month)

  • Work at a reduced time resolution

With instrument and sensor names sorted (with spectral irradiance a special case) the next step is to build a data dictionary called d with keys corresponding to sensor (data) types: conductivity, temperature, pco2 etcetera. The corresponding value for each sensor key is a five-tuple indexed [0], [1], …, [4].

0: XArray DataArray: sensor data
1: XArray DataArray: sensor depth (meters, negative down) corresponding to data
2: float: Default charting lower limit for data
3: float: Default charting upper limit for data
4: string: Default chart color e.g. "blue"

Topic overview#

  • Data: Table below, see also the Data chapter

OOI abbrev

sensors

remark

instrument; sensor abbreviations

ctdpf

5

CTD: includes a variety of sensors

ctd; salinity, temperature, pressure, density, conductivity

pco2

1

carbonate chemistry, midnight and noon descent only

pco2; pco2

phsen

1

pH, midnight and noon descent only

ph; ph

nutnr

2

nitrate, dark samples; midnight and noon ascent only

nitrate; nitrate, …?…

flort

3

Fluorometer triplet: chlorophyll-A, FDOM, backscatter

fluor; chlora, fdom, backscatter

parad

1

photosynthetically available radiation ‘PAR’

par; par

velpt

3

water velocity in three dimensions

vel; north, east, up

spkir

7

spectral irradiance

spkir; spkir412nm, etcetera 443, 490, 510, 555, 620, 683

optaa

86

spectrophotometer ascent noon/midnight

sp; sp00 … sp85

  • Profiler timing metadata

    • Nine daily profiles: 7 regular + 2 special: noon/midnight variants

    • Basic charts: Depth vs { A, B, C, … }

    • Noon/Midnight charts: pCO2 (Descent), pH (Descent), Nitrate (Ascent), Spectral Irradiance (Ascent)

  • Basic charts: Depth vs { A, B, C, … }

  • Noon/Midnight charts: pCO2 (Descent), pH (Descent), Nitrate (Ascent), Spectral Irradiance (Ascent)

  • Turbulence

    • Most direct resource is variation with depth of observed lateral currents (see vel sensor)

  • Comparatives

    • Ascent versus Descent

    • Platform versus Profiler when Profiler is at rest near the Platform

    • Profiler versus Discrete Sample CTD casts during maintenance cruises

  • BioOptics

    • FDOM/CDOM and Chlorophyll

    • Spectral Irradiance

    • Spectrophotometer

  • Visualization in practice

    • Interactivity: Sliders, Widgets

    • Visualizing in 3D

    • Saving charts as .png images

    • Creating chart animations

Developing the data dictionary: sensor profiles#

There are two types of data we are concerned with here: Sensor data and profile metadata. The sensor data from the profiler are viewed in relation to depth, hence ‘profile charts’. In order to generate these we need to know when the profiler is starting and ending an ascent; as well as a descent and a rest period when it is parked on the platform at 200 meters.

Profile metadata is read using the function ReadProfileMetadata(filename) where the filename describes the site and the time period of interest. The data are held in a pandas Dataframe.

The data structure sensors is a list of 2-element lists. The first element is a sensor name and the second is the corresponding instrument that carries the sensor.

sensors[3] = ['salinity', 'ctd']

The data structure ranges is a dictionary of expected upper and lower bounds for each sensor. The keys are sensor names as found in sensors; values are tuples.

ranges['salinity'] = (31, 35)

The data structure standard_deviations is a dictionary of standard deviation ranges, also tuples.

standard_deviations['salinity'] = (.0, .4)

The data structure colors is a dictionary of colors associated with sensors in charts.

colors['salinity'] = 'cyan'

The data structure sensor_names is a dictionary of expanded sensor names more suited to chart labels.

sensor_names['salinity'] = 'Salinity'

from shallowprofiler import *
from charts import *

# profiles is a pandas DataFrame treated as a global resource
# The directory tree for the Jupyter Book:
# 
# book/chapters/rob/<notebooks.ipynb>                    (where this notebook resides)
#              /rca/profiles/osb/january2022.csv
#                           /oos
#                           /axb
#
# The resulting DataFrame has columns r0t, r0z, r1t, r1z, a0t, a0z, a1t, a1z, d0t, d0z, d1t, d1z
#   corresponding to rest / ascent / descent <start - end> <depth> <time>
#
profiles = ReadProfileMetadata()


# Let's look at some typical ascent and descent timestamps
print('First slow ascent start time is local midnight:', profiles['a0t'][3], 'UTC')
print()
print("9 ascent intervals:")
for i in range(9):
    print(profiles['a1t'][i] - profiles['a0t'][i], '         depths:', profiles['a0z'][i], 'to', profiles['a1z'][i], 'meters')
print()
print("9 descent intervals:")
for i in range(9):
    print(profiles['d1t'][i] - profiles['d0t'][i], '         depths:', profiles['d0z'][i], 'to', profiles['d1z'][i], 'meters')
print()
print("9 rest intervals:")
for i in range(9):
    print(profiles['r1t'][i] - profiles['r0t'][i], '         depths:', profiles['r0z'][i], 'to', profiles['r1z'][i], 'meters')
print()

Jupyter Notebook running Python 3
First slow ascent start time is local midnight: 2022-01-01 07:17:00 UTC

9 ascent intervals:
0 days 01:10:00          depths: -191.0 to -19.0 meters
0 days 00:38:00          depths: -194.0 to -105.0 meters
0 days 01:10:00          depths: -196.0 to -24.0 meters
0 days 01:13:00          depths: -194.0 to -12.0 meters
0 days 01:10:00          depths: -193.0 to -23.0 meters
0 days 01:10:00          depths: -199.0 to -29.0 meters
0 days 00:20:00          depths: -203.0 to -163.0 meters
0 days 01:10:00          depths: -197.0 to -21.0 meters
0 days 01:12:00          depths: -191.0 to -12.0 meters

9 descent intervals:
0 days 00:39:00          depths: -19.0 to -194.0 meters
0 days 00:23:00          depths: -105.0 to -196.0 meters
0 days 00:39:00          depths: -24.0 to -196.0 meters
0 days 01:42:00          depths: -12.0 to -192.0 meters
0 days 00:39:00          depths: -23.0 to -198.0 meters
0 days 00:39:00          depths: -29.0 to -203.0 meters
0 days 00:14:00          depths: -163.0 to -202.0 meters
0 days 00:39:00          depths: -21.0 to -192.0 meters
0 days 01:36:00          depths: -12.0 to -191.0 meters

9 rest intervals:
0 days 00:22:00          depths: -191.0 to -191.0 meters
0 days 00:26:00          depths: -194.0 to -194.0 meters
0 days 01:09:00          depths: -196.0 to -196.0 meters
0 days 00:41:00          depths: -196.0 to -194.0 meters
0 days 00:55:00          depths: -192.0 to -193.0 meters
0 days 00:31:00          depths: -198.0 to -199.0 meters
0 days 00:26:00          depths: -203.0 to -203.0 meters
0 days 01:51:00          depths: -202.0 to -197.0 meters
0 days 00:41:00          depths: -192.0 to -191.0 meters

From this we can infer:

  • We are inspecting the nine profile runs from 01-JAN-2022 at the Oregon Slope Base site

  • There are a couple of profiles (2 and 7 in the list of 9) that did not run full scope

    • Ascents stopped below 100 meters so ignore these

  • Profiles 4 and 9 correspond to midnight and noon (slower) profiles

    • Times are UTC: 7 (PST) or 8 (PDT) hours ahead of local

  • Ascent time is about 70 minutes (for both normal and slow profiles)

    • Typical ascent depth range is 174.4 meters

    • Ascent speed is therefore 4.15 cm per second

    • Using an 8-second sample average window would give a vertical sampling of 0.3322 meters / downsample

    • This is about 525 samples per profile

  • Descent time (normal profiles) is about 39 minutes

    • Slow profile descent time is about 100 minutes

  • Rest periods are of less interest

    • They do give an opportunity to track profiler instruments against platform instruments

      • For example: Is a change in salinity reflected in both sensors?

    • Rest times prior to normal profiles seem to vary in duration (under one hour)

    • Rest times prior to midnight/noon profiles seem longer (more than one hour)

    • Rest depths vary by a couple meters start to end as one would expect from tides etcetera