Well, I did not write about the first hackathon, perhaps now, after the second, is the opportunity to tick off three tasks with one article:

1. Why and how did we start this hackathon series?
   
   Snakemake is one of the world’s leading and most versatile workflow management systems to facilitate data analysis (measured by citations for the old and the current paper) – in particular, analysis that is not centred on artificial‑intelligence tools.
   When you have data to analyse, ordinarily you start by checking their quality. You will discard flawed data. You will process the rest and, finally, you will do some fancy statistics and produce some nice figures to show your peers in seminars and research papers your precious results. That is what you do and tools like Snakemake help you to automate this process. This works on any compute environment you want – laptops, servers, of if more power is required supercomputers (a.k.a. HPC systems) or cloud environments.
   
   Almost ten years ago, I started working with Snakemake. I picked it over Nextflow to better support our life science users because the community and —and, in particular, its lead developer— really are approachable and helpful.¹,²
   
   It was during a community call (online video chat) only Johannes Köster (the „inventor“ of Snakemake“) and I were left when I asked him „Considering the amount of pending issues would it not be a good idea to take a whole day off to work through them?“ I wanted to argue, that occasional online meetings are less suited to getting work done. We would need a little more time. And that online-only is less suited to really work with many people on a large code base. I was already dreaming of something bigger, but did not dare to bring up the idea of a full-fledged hackathon just yet.
   Well, Johannes had no objection. On the contrary: He was already nursing a similar idea. A little back and forth and the idea of hosting a hackathon emerged.
   
   This, together with the fact that CERN staff wanted to adapt Snakemake for their purposes (and had a modest grant to support networking events), led us to host the first Snakemake hackathon in Geneva. Over forty participants gathered in a beautiful, windowless room that still bore the odour of four decades without renovation. It was marvellous!
   
2. The Setting
   
   Snakemake started as a tool for bioinformaticians. Meanwhile, folks from various scientific domains adopted it; high-energy physics is only one of them.
   
   So, the participants attending the hackathon came from all over the world (but mostly from Central Europe) and represented a number of communities —chiefly bioinformatics and high‑energy physics. In the end, we had to select for participants with contribution to Snakemake or some footprint in the research software engineering (RSE) world.
   
   Once we arrived, we organized ourselves in interest groups like core development or plugin development or the improvement of websites such as the workflow catalogue. Interest facilitates passion — and productivity in turn.
   Sitting for hours behind a screen does not necessarily boost productivity. So, we went for lunch together, had short strolls and, of course, some seminars. „Seminar“ is a bold word for stand-up talks by volunteers, bit it sounds more academic and everybody can relate to it. It also proved important, as you’ll see later.
   
   At the end of each hackathon we had releases. The first hackathon culminated in a major (breaking) release of Snakemake, introducing numerous new features and bug fixes. My modest SLURM plugin for high‑performance computing clusters gained improved GPU support, and the workflow catalogue underwent a tremendous overhaul, including new documentation on how to contribute.
   
   I do not want to downplay all the other achievements but want to focus on the hackathon itself. To publish those outcomes, we agreed on creating a Research Equals collection. The idea was to give every participant a DOI for every published outcome. As if such incentives were needed! In the end I had to typeset every contribution and check for errors. It was quite a bit of work; honestly, unnecessarily tedious. And it did not pay off. No one will ever cite this work, as doing so requires reading through all those mini-papers not to miss any dependencies.
   
3. Changes from round 1 to 2 to 3
   
   The most obvious change from our first to our second Snakemake hackathon is, of course, the location. We will not be able to host every Snakemake hackathon in Geneva. And we do not want to. This year we met in Munich, courtesy of the Data Science Institute of the Technical University of Munich. This time around was bright, newly fitted, and had windows. And a stunning view onto the university’s abandoned research reactor.
   
   Also, this time we got support from BioHackrXiv to publish our achievements as one preprint. I dare write, it is much more readable and citable – or so I hope albeit the major goal is not to gather citations.
   
   Again, we had „seminars“ of some kind. I prepared a short summary of last year’s development on the SLURM-Executor plugin for HPC systems, for instance. Thought I gather perhaps half a dozen of folks interested in this topic. The room was to small. People were standing³.
   New were discussions (with everyone interested to join) on what to consider for an improved command line interface (which will take some more time to come), a really fancy progress display system (or rather an improved output thereof) and an AI contribution policy (which I still need to link in my issue and PR templates).
   
   Every time, at the end of each hackathon, we took on suggestions for improvements. Last year, Cade Mirchandi (at the time grad student at UC Santa Cruz) and me had the idea of more or less regular online meetings to connect as developers. Eventually, Cade took on the task to organize this as I did not find the time to do it⁴. After the summer break, I hope we will resume our meetings. We actually might open them for questions from users, too.
   
   Another suggestion from this year’s stand-up at the end was to become a bit more formal with the application process: If we were to ask for contributions, e.g. talks, getting funding or support from PIs will ease the convincing some have to do.
   
4. The Future
   
   First, a spoiler: I am hooked and want to keep organizing this hackathon as long as possible. With this you as a reader ought to know, that the entire article and particularly this outlook reflects my personal view, only.
   
   Each time we welcomed new people to the developer community. Each time we were at a different location. It looks like we will meet at a different location yet again (next year). Of course, we will never be able to guarantee changing locations. My guess is we will need to confine ourselves to Europe for the time being – currently, hosting scientific events on US soil is no option. And finding sponsors to venture further away is hard⁵.
   
   A hackathon like this one resembles an unconference model like the one we use for Open Science retreats (blog1, blog2) inasmuch as the participants propose and choose their work items for the hackathon days. This leads to some really crazy and awesome ideas. Some op them certainly will be a topic for upcoming blog articles.
   
   I think the HPC world, in which I work, is somewhat prejudiced against life sciences (you might want to read between the lines, here). Having the teams from the CERN on board together with their future processing needs will certainly help the cause for future Snakemake support on HPC clusters. This particular community has enormous HPC requirements after all.
   
   The hackathon, I hope, will help all researchers interested in automated, fast, reliable and reproducible computing. As a maintainer, I will do my best, too.
   

1. I do consider the Nextflow community to be a friendly competition. They certainly are not less friendly than the Snakemake community. It’s just that a single soul can only do that much work, which is why I focussed on just one system in terms of development. Support regarding deployment for users is a different matter. ↩︎
2. Yes, there are other workflow management systems. But none are so generic as these two (some focus on Cloud systems, some confine themeselves to certain scientific fields, some require CWL translation layers, etc.). Then there is Galaxy, another workflow management system popular in bioinformatics. It takes relatively more effort to launch on HPC systems which are my focus, though. Generally, I do not mind, if people use a different one – as long as they try to live up to the ideas of reproducibility and transparency. A bit of competition will not harm us. ↩︎
3. And I really appreciate the positive feedback! ↩︎
4. Thank you, Cade! ↩︎
5. I would not know whether I would want to participate in any scientific event too far away. Flying, for me, is no option in times of global warming. I will not condemn anyone who flies, but I would not be able to reconcile flying with my own conscience. ↩︎