Convert YAXArrays

This section describes how to convert variables from types of other Julia packages into YAXArrays and vice versa.

WARNING

YAXArrays is designed to work with large datasets that are way larger than the memory. However, most types are designed to work in memory. Those conversions are only possible if the entire dataset fits into memory. In addition, metadata might be lost during conversion.

Convert Base.Array

Convert Base.Array to YAXArray:

using YAXArrays

m = rand(5,10)
a = YAXArray(m)

╭──────────────────────────╮
│ 5×10 YAXArray{Float64,2} │
├──────────────────────────┴──────────────────────────────────── dims ┐
  ↓ Dim_1 Sampled{Int64} Base.OneTo(5) ForwardOrdered Regular Points,
  → Dim_2 Sampled{Int64} Base.OneTo(10) ForwardOrdered Regular Points
├─────────────────────────────────────────────────────────── metadata ┤
  Dict{String, Any}()
├────────────────────────────────────────────────────────── file size ┤ 
  file size: 400.0 bytes
└─────────────────────────────────────────────────────────────────────┘

Convert YAXArray to Base.Array:

m2 = collect(a.data)

5×10 Matrix{Float64}:
 0.0479824  0.21733   0.503822   0.471616   …  0.727172  0.859895   0.835692
 0.313251   0.690378  0.276406   0.0524268     0.580563  0.799858   0.794808
 0.261339   0.248805  0.0272231  0.482603      0.435339  0.817619   0.459364
 0.253175   0.218575  0.516812   0.816341      0.112915  0.0898559  0.628495
 0.347648   0.411819  0.65062    0.866521      0.365944  0.929291   0.693457

Convert Raster

A Raster as defined in Rasters.jl has a same supertype of a YAXArray, i.e. AbstractDimArray, allowing easy conversion between those types:

using Rasters

lon, lat = X(25:1:30), Y(25:1:30)
time = Ti(2000:2024)
ras = Raster(rand(lon, lat, time))
a = YAXArray(dims(ras), ras.data)

╭────────────────────────────╮
│ 6×6×25 YAXArray{Float64,3} │
├────────────────────────────┴────────────────────────── dims ┐
  ↓ X  Sampled{Int64} 25:1:30 ForwardOrdered Regular Points,
  → Y  Sampled{Int64} 25:1:30 ForwardOrdered Regular Points,
  ↗ Ti Sampled{Int64} 2000:2024 ForwardOrdered Regular Points
├─────────────────────────────────────────────────── metadata ┤
  Dict{String, Any}()
├────────────────────────────────────────────────── file size ┤ 
  file size: 7.03 KB
└─────────────────────────────────────────────────────────────┘

ras2 = Raster(a)

╭──────────────────────────╮
│ 6×6×25 Raster{Float64,3} │
├──────────────────────────┴──────────────────────────── dims ┐
  ↓ X  Sampled{Int64} 25:1:30 ForwardOrdered Regular Points,
  → Y  Sampled{Int64} 25:1:30 ForwardOrdered Regular Points,
  ↗ Ti Sampled{Int64} 2000:2024 ForwardOrdered Regular Points
├─────────────────────────────────────────────────── metadata ┤
  Dict{String, Any}()
├───────────────────────────────────────────────────── raster ┤
  extent: Extent(X = (25, 30), Y = (25, 30), Ti = (2000, 2024))

└─────────────────────────────────────────────────────────────┘
[:, :, 1]
  ↓ →  25          26         27         28         29          30
 25     0.632948    0.68686    0.705456   0.960694   0.671561    0.166385
 26     0.598948    0.844498   0.783346   0.142439   0.0439638   0.896142
 27     0.130873    0.716444   0.414494   0.459177   0.691192    0.00800633
 28     0.0337353   0.469808   0.263889   0.873889   0.432649    0.723609
 29     0.940098    0.83113    0.6651     0.813735   0.0502287   0.851757
 30     0.578336    0.798949   0.72069    0.741497   0.935314    0.972381

Convert DimArray

A DimArray as defined in DimensionalData.jl has a same supertype of a YAXArray, i.e. AbstractDimArray, allowing easy conversion between those types.

Convert DimArray to YAXArray:

using DimensionalData
using YAXArrayBase

dim_arr = rand(X(1:5), Y(10.0:15.0), metadata = Dict{String, Any}())
a = yaxconvert(YAXArray, dim_arr)

╭─────────────────────────╮
│ 5×6 YAXArray{Float64,2} │
├─────────────────────────┴────────────────────────────────── dims ┐
  ↓ X Sampled{Int64} 1:5 ForwardOrdered Regular Points,
  → Y Sampled{Float64} 10.0:1.0:15.0 ForwardOrdered Regular Points
├──────────────────────────────────────────────────────── metadata ┤
  Dict{String, Any}()
├─────────────────────────────────────────────────────── file size ┤ 
  file size: 240.0 bytes
└──────────────────────────────────────────────────────────────────┘

Convert YAXArray to DimArray:

dim_arr2 = yaxconvert(DimArray, a)

╭─────────────────────────╮
│ 5×6 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────── dims ┐
  ↓ X Sampled{Int64} 1:5 ForwardOrdered Regular Points,
  → Y Sampled{Float64} 10.0:1.0:15.0 ForwardOrdered Regular Points
├──────────────────────────────────────────────────────── metadata ┤
  Dict{String, Any}()
└──────────────────────────────────────────────────────────────────┘
 ↓ →  10.0       11.0       12.0       13.0       14.0       15.0
 1     0.881921   0.541503   0.340663   0.575957   0.433296   0.602085
 2     0.73388    0.244494   0.559203   0.556982   0.393411   0.880507
 3     0.647451   0.450589   0.603671   0.681068   0.536156   0.994356
 4     0.148048   0.51328    0.182729   0.577165   0.175644   0.0470657
 5     0.897744   0.898278   0.279639   0.235205   0.395856   0.0687852

INFO

At the moment there is no support to save a DimArray directly into disk as a NetCDF or a Zarr file.