Harald Klinke<p>Offene KI braucht offene Infrastruktur – auch bei der Entwicklung.<br>Das Projekt f13 auf OpenCode stellt Tools für transparente, reproduzierbare Workflows zur Verfügung – von Modelltraining bis Deployment.<br>Ein Schritt Richtung digitaler Souveränität durch offene Werkzeuge.<br><a href="https://det.social/tags/OpenSourceAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenSourceAI</span></a> <a href="https://det.social/tags/Reproducibility" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Reproducibility</span></a> <a href="https://det.social/tags/DigitaleSouver%C3%A4nit%C3%A4t" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DigitaleSouveränität</span></a></p><p><a href="https://gitlab.opencode.de/f13" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">gitlab.opencode.de/f13</span><span class="invisible"></span></a></p>
#reproducibility
3 posts3 participants0 posts today
Retractions and failures to replicate are signs of weak research. But they're also signs of laudable and necessary efforts to identify weak research and improve future research. The #Trump admin is systematically weaponizing these efforts to cast doubt on science as such.
"Research-integrity sleuths say their work is being ‘twisted’ to undermine science."
https://www.nature.com/articles/d41586-025-02163-z
www.nature.comResearch-integrity sleuths say their work is being ‘twisted’ to undermine scienceSome sleuths fear that the business of cleaning up flawed studies is being weaponized against science itself.
Replied in thread
@elduvelle Sure - let’s add to the problem of reproducibility (retraction rates are already much higher for high-impact journals, incl. Nature – Brembs et al 2013 "Deep impact: unintended consequences of journal rank." Frontiers in human Neuroscience) by adding AI peer reviewers and watch academic publishing enshittify further. #reproducibility #impactfactor #ScientificJournals
This bluesky thread by Mark Hanson describes a massive project to look at #reproducibility of drosophila studies - I don't know much about that area, but the approach is pretty exciting and has potential for other topics. https://bsky.app/profile/hansonmark.bsky.social/post/3ltlvkkxyak2k
Bluesky Social · Mark A. Hanson (@hansonmark.bsky.social)The "reproducibility crisis" in science constantly makes headlines. Repro efforts are often limited. What if you could assess reproducibility of an entire field?
That's what @brunolemaitre.bsky.social et al. have done. Fly immunity is highly replicable & offers lessons for #metascience
A 🧵 1/n
The killer quote! Prestigious institutions and prestigious journals drive irreproducibility in the life sciences - well, at least in this particular sample.
And yet another one in the ever increasing list of analyses showing that top journals are bad for science:
"Thus, our analysis show major claims published in low-impact journals are significantly more likely to be reproducible than major claims published in trophy journals. "
bioRxiv · A retrospective analysis of 400 publications reveals patterns of irreproducibility across an entire life sciences research fieldThe ReproSci project retrospectively analyzed the reproducibility of 1006 claims from 400 papers published between 1959 and 2011 in the field of Drosophila immunity. This project attempts to provide a comprehensive assessment, 14 years later, of the replicability of nearly all publications across an entire scientific community in experimental life sciences. We found that 61% of claims were verified, while only 7% were directly challenged (not reproducible), a replicability rate higher than previous assessments. Notably, 24% of claims had never been independently tested and remain unchallenged. We performed experimental validations of a selection of 45 unchallenged claim, that revealed that a significant fraction (38/45) of them is in fact non-reproducible. We also found that high-impact journals and top-ranked institutions are more likely to publish challenged claims. In line with the reproducibility crisis narrative, the rates of both challenged and unchallenged claims increased over time, especially as the field gained popularity. We characterized the uneven distribution of irreproducibility among first and last authors. Surprisingly, irreproducibility rates were similar between PhD students and postdocs, and did not decrease with experience or publication count. However, group leaders, who had prior experience as first authors in another Drosophila immunity team, had lower irreproducibility rates, underscoring the importance of early-career training. Finally, authors with a more exploratory, short-term engagement with the field exhibited slightly higher rates of challenged claims and a markedly higher proportion of unchallenged ones. This systematic, field-wide retrospective study offers meaningful insights into the ongoing discussion on reproducibility in experimental life sciences
### Competing Interest Statement
The authors have declared no competing interest.
Swiss National Science Foundation, 310030_189085
ETH-Domains Open Research Data (ORD) Program (2022)
To my knowledge, first time that not only prestigious journals, but also prestigious institutions are implicated as major drivers of irreproducibility:
"Higher representation of challenged claims in trophy journals and from top universities"
bioRxiv · A retrospective analysis of 400 publications reveals patterns of irreproducibility across an entire life sciences research fieldThe ReproSci project retrospectively analyzed the reproducibility of 1006 claims from 400 papers published between 1959 and 2011 in the field of Drosophila immunity. This project attempts to provide a comprehensive assessment, 14 years later, of the replicability of nearly all publications across an entire scientific community in experimental life sciences. We found that 61% of claims were verified, while only 7% were directly challenged (not reproducible), a replicability rate higher than previous assessments. Notably, 24% of claims had never been independently tested and remain unchallenged. We performed experimental validations of a selection of 45 unchallenged claim, that revealed that a significant fraction (38/45) of them is in fact non-reproducible. We also found that high-impact journals and top-ranked institutions are more likely to publish challenged claims. In line with the reproducibility crisis narrative, the rates of both challenged and unchallenged claims increased over time, especially as the field gained popularity. We characterized the uneven distribution of irreproducibility among first and last authors. Surprisingly, irreproducibility rates were similar between PhD students and postdocs, and did not decrease with experience or publication count. However, group leaders, who had prior experience as first authors in another Drosophila immunity team, had lower irreproducibility rates, underscoring the importance of early-career training. Finally, authors with a more exploratory, short-term engagement with the field exhibited slightly higher rates of challenged claims and a markedly higher proportion of unchallenged ones. This systematic, field-wide retrospective study offers meaningful insights into the ongoing discussion on reproducibility in experimental life sciences
### Competing Interest Statement
The authors have declared no competing interest.
Swiss National Science Foundation, 310030_189085
ETH-Domains Open Research Data (ORD) Program (2022)
Starting your journey in scientific research?
Join the beginner-friendly tutorial at #EuroSciPy2025:
Managing Scientific Data and Workflows with DataLad
Led by Ole Bialas & Michał Szczepanik
In this hands-on session, you'll learn:
How to organize & track your data How to repeat your experiments reliably How to share your results with others
No Git experience required—just curiosity and Python basics!
https://euroscipy.org/schedule
#FAIRdata #Reproducibility #Python
euroscipy.orgEuroSciPy 2025The EuroSciPy meeting is a cross-disciplinary gathering focused on the use and development of the Python language in scientific research.
Tips for using Jupyter notebooks as part of a reproducible workflow (one that goes from raw data to research article with a single command): https://docs.calkit.org/notebooks
docs.calkit.orgNotebooks - Calkit
We invite staff and students at the University of #Groningen to share how they are making #research or #teaching more open, accessible, transparent, or reproducible, for the 6th annual #OpenResearch Award.
Looking for inspiration?
Explore the case studies submitted in previous years:
https://www.rug.nl/research/openscience/open-research-award/previous-events
More info: https://www.rug.nl/research/openscience/open-research-award/
#OpenScience #OpenEducation #OpenAccess #Reproducibility
@oscgroningen
Continued thread
Jack Taylor is now presenting a new #Rstats package: "LexOPS: A Reproducible Solution to Stimuli Selection". Jack bravely did a live demonstration based on a German corpus ("because we're in Germany") that generated matched stimuli that certainly made the audience giggle... let's just say that one match involved the word "Erektion"... 
There is a paper about the LexOPS package: https://link.springer.com/article/10.3758/s13428-020-01389-1 and a detailed tutorial: https://jackedtaylor.github.io/LexOPSdocs/index.html. Also a #Shiny app for those who really don't want to use R, but that allows code download for #reproducibility: https://jackedtaylor.github.io/LexOPSdocs/ Really cool and useful project! #WoReLa1 #linguistics #psycholinguistics
#statstab #378 Selective Inference: The Silent Killer of Replicability
Thoughts: Benjamin overviews the replicability crisis, alternatives to p-values (and their issues), and suggests selective reporting is a large issue itself.
#replication #crisis #pvalue #nhst #selectivereporting #replicability #reproducibility
Harvard Data Science Review · Selective Inference: The Silent Killer of Replicability
Continued thread
Just pushed some updates. I have posted the #verilog serial link code. This involved getting a set of scripts for simulation set up.
If anybody is interested in helping on this project, I'd love to have somebody try to follow the readme.md to install the prerequisites and run the ./run_sim script.
The goal would be to take notes and flesh out what things aren't in the prerequisite list and how to install them.
I'd be happy to do the same for somebody else's project.
To ensure AI is genuinely open source, we must have complete access to:
1. The datasets used for its training and evaluation
2. The underlying code
3. The design of the model
4. The model's parameters
Without these elements, transparency and the ability to replicate results remain incomplete.