My mental picture of image files has always been of pixels covering a surface as tiles each like a tiny rectangular shapefile.
Investigating #Python #xarray has made me see the elegance of handling images as a grid of equally spaced dimensionless sensor readings. Upscaling/downscaling and interpolation become more meaningful and lossless, and image data is functionally identical to (although denser than) other point-based sensor data (e.g. weather stations).
The data science becomes so clean.
Excellent #OpenSource #OpenScience job opportunity: #xarray community developer at Earthmover PBC:
https://github.com/pydata/xarray/discussions/9059
US-only , but remote available
. And the Earthmover folks are awesome!
v0.3.0 of yt_xarray is out!
This includes the initial release of the embedded transformation framework -- the main perk being how it simplifies the process of using yt's volume rendering methods with non-cartesian data (which is common in geophysical datasets)!
Full release notes at https://github.com/data-exp-lab/yt_xarray/releases/tag/v0.3.0
Figure uses relative humidity from a MERRA-2, see https://chrishavlin.github.io/NASASoftwareWorkshop2024/yt-xr-dev-02-transformations.html for an overview.
Checking in from New Zealand! I'm developing a library to decode #GeoTIFF files in #rustlang (without using GDAL ). Currently trying to add an #xarray backend (in Python) at https://github.com/weiji14/cog3pio/pull/14.
Stoked about recent progress on yt_xarray's embedded transformation framework! This new feature takes a non-cartesian #xarray dataset and wraps it in a cartesian #yt dataset. As yt needs data, it will interpolate data on demand. This let's you use yt's volume rendering without saving off interpolated versions of datasets (which is how I used to do things...). Makes it **much** easier to generate volume renderings of geophysical fields with yt.
#python #data_viz #3d
I have ace colleagues, part 243:
My colleague Luke (LukeDataManager) has a Youtube channel where he helps people prepare #OpenData #FAIRdata:
His most recent series is about #Python, #Xarray, #NetCDF, etc...
Give him a follow (on Youtube) and a retoot (here)? https://youtube.com/@LukeDataManager?si=VMU9ehIPE8eyRaCt
@asm Thank you for introducing me to Xarray (https://github.com/pydata/xarray) ! Like #Pandas but faster simpler and more scalable. It even has labeled dimensions, like a #DataFrame. Looks like it's compatible with #Dask too.
Py3DEP [hydroclimate analysis]
--
https://pypi.org/project/py3dep/ <-- link to app / resources
--
“Py3DEP is a part of [PyGeoUtils] HyRiver software stack that is designed to aid in hydroclimate analysis through web services. This package provides access to the 3DEP database which is a part of the National Map services. The 3DEP service has multi-resolution sources and depending on the user-provided resolution, the data is resampled on the server-side based on all the available data sources. Py3DEP returns the requests as xarray dataset…”
#GIS #spatial #mapping #Py3DEP #PyGeoUtils #HyRiver #softwarestack #elevation #3DEP #USGS #NationalMap #opendata #processing #spatialanalysis #hydroclimate #water #hydrology #webservices #xarray
@HyRiver @USGS
#DYK?
One can now open #netCDF files having groups with #python #xarray-datatree !
> datatree.open_datatree('file.nc')
-> https://xarray-datatree.readthedocs.io/en/latest/index.html
So far I had to handle dict() of xarray datasets and it was a pain. Looking forward to giving this a try with actual satellite data files!
prototyping some work for yt_xarray!
working on a coordinate transformation pipeline that let's you build a cartesian yt dataset that samples data that is natively defined in a different coordinate system on the fly. This let's you use yt's full volume rendering (or in this video's case yt_idv's interactive rendering) without pre-computing an interpolated dataset. Instead, the interpolation happens within yt's chunking framework only when it needs it.
Presenting my talk today on "The ecosystem of geospatial machine learning tools in the Pangeo world"
Slides: https://hackmd.io/@weiji14/foss4g2023oceania
Code: https://github.com/weiji14/foss4g2023oceania
#FOSS4GOceania2023 #FOSS4G #Pangeo #GeoML @zarr #xarray #xbatcher #zen3geo #gischat
I have just discovered that #xarray has a weighted DataArray and this is an absolute game changer! The more I use xarray the more I appreciate what an awesome library it is.