{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Basic functionality of octant package" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Import the necessary modules" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "from octant.core import TrackRun, OctantTrack, HOUR" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Define the common data directory" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "sample_dir = Path(\".\") / \"sample_data\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Data are usually organised in hierarchical directory structure. Here, the relevant parameters are defined." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "dataset = \"era5\"\n", "period = \"test\"\n", "run_id = 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Construct the full path" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "track_res_dir = sample_dir / dataset / f\"run{run_id:03d}\" / period" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load the data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The standard loading procedure is done during the initialisation of `TrackRun` by reading text files (output from PMCTRACK) from the given directory." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "tr = TrackRun(track_res_dir)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "[671 tracks]\n", "\n", "Data columns:\n", "lon | lat | vo | time | area | vortex_type\n", "\n", "Sources:\n", "sample_data/era5/run000/test\n" ] } ], "source": [ "print(tr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `TrackRun` object also has an HTML view available in Jupyter Notebooks" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Cyclone tracking results
Number of tracks671
Data columnslon, lat, vo, time, area, vortex_type
Sources
sample_data/era5/run000/test
\n", " " ], "text/plain": [ " [671 tracks]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "octant.core.TrackRun" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(tr)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "671" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(tr)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "671" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr.size()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1.0" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr.tstep_h" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Main data container" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The main attribute of `TrackRun` is `.data`, which has all the tracks stored in one DataFrame-like object." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
lonlatvotimeareavortex_type
track_idxrow_idx
009.379.20.0003452011-01-01 00:00:0010617.936520
19.379.20.0003522011-01-01 01:00:0010154.146480
29.379.20.0003622011-01-01 02:00:0010136.946290
1020.470.50.0003172011-01-01 00:00:0034750.660161
120.770.50.0003102011-01-01 01:00:0036727.843751
\n", "
" ], "text/plain": [ " lon lat vo time area \\\n", "track_idx row_idx \n", "0 0 9.3 79.2 0.000345 2011-01-01 00:00:00 10617.93652 \n", " 1 9.3 79.2 0.000352 2011-01-01 01:00:00 10154.14648 \n", " 2 9.3 79.2 0.000362 2011-01-01 02:00:00 10136.94629 \n", "1 0 20.4 70.5 0.000317 2011-01-01 00:00:00 34750.66016 \n", " 1 20.7 70.5 0.000310 2011-01-01 01:00:00 36727.84375 \n", "\n", " vortex_type \n", "track_idx row_idx \n", "0 0 0 \n", " 1 0 \n", " 2 0 \n", "1 0 1 \n", " 1 1 " ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr.data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It always has `track_idx` level in order to index through individual tracks." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tracking run configuration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If a `.conf` file is present in the same directory, it is loaded to `TrackRun`." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Tracking algorithm settings (43)
dt_start =201101010000
dt_end =201101312300
vor_lvl =950
steer_lvl_btm =1000
steer_lvl_top =700
datadir =../../reanalysis/era5
outdir =../results/test7
vort_name =vo
u_name =u
v_name =v
psea_name =msl
land_name =lsm
prefix_lvl =era5.an.pl.
prefix_sfc =era5.an.sfc.
proj =1
vert_grid =1
lon1 =-20
lon2 =50
lat1 =65
lat2 =85
halo_r =30.0
smth_type =0
nsmth_x =10
nsmth_y =10
r_smth =60.0
zeta_max0 =0.0002
zeta_min0 =0.00015
int_zeta_min0 =2e-06
gamma =0.25
d_cf_min =400.0
size_synop =90000.0
distance_ec =300.0
del_psea_min =0.5
steering_type =2
n_steering_x =20
n_steering_y =20
r_steering =200.0
track_type =2
del_lon =1.0
del_lat =0.8
del_r =120.0
merge_opt =2
vor_out_on =0
\n", " " ], "text/plain": [ "Tracking algorithm settings (43)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr.conf" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Other methods of initialisation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`TrackRun` class can be initialised empty:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "tr_empty = TrackRun()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Cyclone tracking results
Number of tracks0
Data columnslon, lat, vo, time, area, vortex_type
\n", " " ], "text/plain": [ " [0 tracks]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr_empty" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Concatenate data from several directories" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "TR = TrackRun(one_directory)\n", "TR2 = TrackRun(another_directory)\n", "TR.extend(TR2)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "or" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "TR += TrackRun(another_directory)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Some attributes of TrackRun" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`TrackRun` object has a few useful properties" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* for example, it is possible to access the list of files that the data were loaded from. " ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[PosixPath('sample_data/era5/run000/test/vortrack_0001_0001.txt'),\n", " PosixPath('sample_data/era5/run000/test/vortrack_0002_0001.txt'),\n", " PosixPath('sample_data/era5/run000/test/vortrack_0003_0001.txt'),\n", " PosixPath('sample_data/era5/run000/test/vortrack_0004_0001.txt'),\n", " PosixPath('sample_data/era5/run000/test/vortrack_0005_0001.txt')]" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr.filelist[:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* there is a shortcut to group tracks by their index" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr.gb" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is equivalent of doing `tr.data.groupby(\"track_idx\")`." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Track with index=0\n", " lon lat vo time area \\\n", "track_idx row_idx \n", "0 0 9.3 79.2 0.000345 2011-01-01 00:00:00 10617.93652 \n", " 1 9.3 79.2 0.000352 2011-01-01 01:00:00 10154.14648 \n", " 2 9.3 79.2 0.000362 2011-01-01 02:00:00 10136.94629 \n", "\n", " vortex_type \n", "track_idx row_idx \n", "0 0 0 \n", " 1 0 \n", " 2 0 \n" ] } ], "source": [ "for idx, a_track in tr.gb:\n", " print(f\"Track with index={idx}\")\n", " print(a_track)\n", " break" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* current `TrackRun` is not categorised yet (see \"Categorisation\" examples), so this attribute is empty:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr.cat_labels" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Serialising TrackRun" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`TrackRun` and all its metadata can be saved to and loaded from an HDF file." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "tr.to_archive(\"test.h5\")" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "new_tr = TrackRun.from_archive(\"test.h5\")" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Cyclone tracking results
Number of tracks671
Data columnslon, lat, vo, time, area, vortex_type
Sources
sample_data/era5/run000/test
\n", " " ], "text/plain": [ " [671 tracks]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "new_tr" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Units of TrackRun" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "import random" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each cyclone track stored in lists of `TrackRun` class as a `OctantTrack` instance" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "ot = random.choice([*tr[:].gb])[1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is essentially a sub-class of pandas.DataFrame" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
lonlatvotimeareavortex_type
track_idxrow_idx
124012.372.00.0004232011-01-05 23:00:007615.172851
110.872.00.0003932011-01-06 00:00:0019943.687501
28.772.00.0003492011-01-06 01:00:0018887.552731
3-3.667.20.0006662011-01-06 02:00:0095968.203123
\n", "
" ], "text/plain": [ " lon lat vo time area \\\n", "track_idx row_idx \n", "124 0 12.3 72.0 0.000423 2011-01-05 23:00:00 7615.17285 \n", " 1 10.8 72.0 0.000393 2011-01-06 00:00:00 19943.68750 \n", " 2 8.7 72.0 0.000349 2011-01-06 01:00:00 18887.55273 \n", " 3 -3.6 67.2 0.000666 2011-01-06 02:00:00 95968.20312 \n", "\n", " vortex_type \n", "track_idx row_idx \n", "124 0 1 \n", " 1 1 \n", " 2 1 \n", " 3 3 " ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ot" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "octant.core.OctantTrack" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(ot)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It has a few useful properties" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "3.0" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ot.lifetime_h" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "836.7043121073503" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ot.total_dist_km" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "including maximum vorticity in $ s^{-1}$:" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.00066578" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ot.max_vort" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "810.8722660749592" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ot.gen_lys_dist_km" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "278.90143736911676" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ot.average_speed" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[12.3, 72. ],\n", " [10.8, 72. ],\n", " [ 8.7, 72. ],\n", " [-3.6, 67.2]])" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ot.lonlat" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Running octant with a progress bar" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Having either the `fastprogress` or `tqdm` module installed allows for running some methods with a bar that shows progress. To enable it, set the attribute of `RUNTIME` variable to `True`:" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "import octant\n", "octant.RUNTIME.enable_progress_bar = True" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "conditions = [\n", " (\"long_lived\", [lambda ot: ot.lifetime_h >= 6]),\n", " (\n", " \"far_travelled_and_very_long_lived\",\n", " [lambda ot: ot.lifetime_h >= 36, lambda ot: ot.gen_lys_dist_km > 300.0],\n", " ),\n", " (\"strong\", [lambda x: x.max_vort > 1e-3]),\n", "]" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " \n", " 100.00% [671/671 00:02<00:00]\n", "
\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "tr.classify(conditions)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "More `classify()` [examples](01_Categorisation.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## octant's utilities" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "from octant.utils import great_circle" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "222826.50759451024" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "great_circle(lon1=9.6, lon2=10.2, lat1=76.9, lat2=78.9)" ] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:test_env]", "language": "python", "name": "conda-env-test_env-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 4 }