Recent searches

No recent searches

Search options

Only available when logged in.
mastodon.ie is one of the many independent Mastodon servers you can use to participate in the fediverse.
Irish Mastodon - run from Ireland, we welcome all who respect the community rules and members.

Administered by:

Server stats:

1.5K
active users

mastodon.ie: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.3

#snakemakehackathon2025

0 posts0 participants0 posts today
Christian Meesters<p><span class="h-card" translate="no"><a href="https://mstdn.science/@dryak" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>dryak</span></a></span> returning from the <a href="https://fediscience.org/tags/isc25" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>isc25</span></a> I will continue this thread with something applicable everywhere, not just on <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HPC</span></a> clusters:</p><p>Workflow runs can crash. There are a number of possible reasons. Snakemake offers a `--rerun-incomple` flag (or short `--ri`) which lets a user resume a workflow.</p><p>This contribution from Filipe G. Viera describes a small fix to stabilize the feature. Not only will incomplete files be removed after a crash, now it is ensured that all metadata with them are deleted too, before resuming: <a href="https://zenodo.org/records/15490098" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">zenodo.org/records/15490098</span><span class="invisible"></span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Today tooting from the <a href="https://fediscience.org/tags/ISC25" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ISC25</span></a> - the International Supercomputing Conference. What better opportunity to brag about something I've done to facilitate using GPUs with Snakemake?</p><p>Here is my contribution, simpler job configuration for GPU jobs:</p><p><a href="https://doi.org/10.5281/zenodo.15551797" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">doi.org/10.5281/zenodo.1555179</span><span class="invisible">7</span></a></p><p>Not alone though: Without valuable input of <span class="h-card" translate="no"><a href="https://mstdn.science/@dryak" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>dryak</span></a></span> . Without him, I would have overlooked something crucial.</p><p>And when we talk about reproducible AI, my take is that we ought to consider workflow managers, too. Something which protocols what you have done with little effort.</p><p><a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Before I continue uploading - and I do have a couple of more contributions to add to the <a href="https://fediscience.org/tags/ResearchEquals" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ResearchEquals</span></a> collection - first another contribution by Johanna Elena Schmitz and Jens Zentgraf made at the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> </p><p>One difficulty when dealing with a different scientific question: Do I need to re-invent the wheel (read: write a Workflow from scratch?) just to address my slightly different question?</p><p>Snakemake already allowed to incorporate "alien" workflows, even <a href="https://fediscience.org/tags/Nextflow" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Nextflow</span></a> workflows, into desired workflows. The new contribution allows for a more dynamic contribution - with very little changes.</p><p>Check it out: <a href="https://zenodo.org/records/15489694" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">zenodo.org/records/15489694</span><span class="invisible"></span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Let's take a look at another contribution of Johanna Elena Schmitz and Jens Zentgraf from the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> </p><p>Snakemake users probably know that </p><p>`$ snakemake [args] --report`</p><p>will generate a self-contained HTML report. Including all plots and <a href="https://fediscience.org/tags/metadata" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>metadata</span></a> a researcher's heart longs for.</p><p>Now, why trigger this manually? If the workflow runs successfully, now we can write (or configure):</p><p>`$ snakemake [args] --report-after-run`</p><p>and Snakemake will autogenerate the same report.</p><p>For details see <a href="https://doi.org/10.5281/zenodo.15489764" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">doi.org/10.5281/zenodo.1548976</span><span class="invisible">4</span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <br><a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>One important feature implemented in the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> : Snakemake will calculate file checksums to detect changes. If a file changes, the rule producing it needs to be re-executed when a workflow it re-triggered. But what if a file is too big for reasonable checksum calculation? You do not what to wait forever, after all.</p><p>This contribution describes the implementation of a threshold users may set: <a href="https://doi.org/10.5281/zenodo.15489401" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">doi.org/10.5281/zenodo.1548940</span><span class="invisible">1</span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>One important bug fix during the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> : the config replacement. Now, users can overwrite existing configurations entirely with `--replace-workflow-config`.</p><p>Details: <a href="https://zenodo.org/records/15479268" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">zenodo.org/records/15479268</span><span class="invisible"></span></a></p><p>More at <a href="https://www.researchequals.com/collections/hm1w-cg/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">researchequals.com/collections</span><span class="invisible">/hm1w-cg/</span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/openscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>openscience</span></a></p>
Christian Meesters<p>Did you know? During the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> we had a staggering 194 work items!</p><p>It took a while, but now we are gathering contribution reports and present them online as a ResearchEquals (<a href="https://fediscience.org/@ResearchEquals@akademienl.social" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">fediscience.org/@ResearchEqual</span><span class="invisible">s@akademienl.social</span></a>) collection:</p><p><a href="https://www.researchequals.com/collections/hm1w-cg" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">researchequals.com/collections</span><span class="invisible">/hm1w-cg</span></a></p><p>The first 10 are online and I will post some highlights in the coming weeks.</p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleResearch</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Busy year:</p><p>- Workflow programming for Data Analysis on <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HPC</span></a> Systems (Course in Mainz in January): ✅ <br>- Same Course in Dresden (February) ✅ <br>- <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> at the CERN in March: ✅ <br>- upcoming: <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a> Retreat (no hashtag, yet?) in April<br>- International Supercomputing Conference in June (so, <span class="h-card" translate="no"><a href="https://mast.hpc.social/@boegel" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>boegel</span></a></span>, I will be there, after all and hope to meet people from <span class="h-card" translate="no"><a href="https://fosstodon.org/@irods" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>irods</span></a></span>, too ; will you be there folks from <a href="https://fediscience.org/tags/iRODS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>iRODS</span></a> ?)<br>- German Conference for <a href="https://fediscience.org/tags/Bioinformatics" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Bioinformatics</span></a> and NHR Conference in September</p><p>And I do not know whether this will be all. I have a nagging feeling there is more to come 😉</p>
Christian Meesters<p>Today at the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a>:</p><p>We did a tour around the CERN!</p><p>But, of course, work was done, too! On cvmfs support for software deployment, many core features, improved metadata reporting, more stable SLURM executor, etc. etc. etc. Soon you will read more. 😉</p>
Johannes Köster<p>The second day of the <a href="https://fosstodon.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> is over. Today I have merged 11 fantastic pull requests in the in main snakemake repo alone. And we have our first <a href="https://fosstodon.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> release during the hackathon, adding integration with xonsh for scripting and helpers for extracting params from input files.</p>
Christian Meesters<p><a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> ! We started!</p><p>At the CERN for better <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> and <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleResearch</span></a> .</p>
TrendingLive feeds
About