Wodden attic with empty shelves

This release contains some template changes and bug fixes.

Highlights

We refactored the pipelines template a bit:

  • The lib directory is removed: 👋 Groovy code
  • Instead we now use nf-core subworkflows for pipeline initialisation
  • The nf-validation plugin is now used to create an input channel from a sample sheet file.
Tip

If you haven’t started merging in the template updates yet, it may almost be easier to ignore the template updates for now and try and remove the lib/ directory on the pipeline dev branch by:

  1. Individually installing the utils_* subworkflows with the following commands:
nf-core subworkflows install utils_nextflow_pipeline
nf-core subworkflows install utils_nfcore_pipeline
nf-core subworkflows install utils_nfvalidation_plugin
  1. Creating a local utils_* subworkflow for the pipeline. You can copy the one in rnaseq (dev) or the pipeline template and customise this to your requirements. Make sure you move any custom functions in lib/ directory to this file.
  2. Include the utils_* subworkflows in the main.nf as done in rnaseq (dev).
  3. Including the utils_* subworkflows in the workflow main.nf as done in rnaseq (dev).
  4. Delete the lib/ directory after you have confirmed everything has been ported across.
  5. Once you have merged this to dev the template sync PR will be updated and tell you whether you have missed anything.
  6. The nf-core lint command might complain about having to recompute checksum of subworkflow(s).
    • Be sure to check in the modules.json file that non of the previously installed subworkflows have dissapeared from it.
    • A new subworkflows section with the new utils_* subworkflows might have been added during the merged. Reinstalling the subworkflows using nf-core subworkflows install should fix this, otherwise one could manually edit the modules.json file (only recommended for advanced users).

💡 It helped to disable running the main workflow whilst wiring all of this in to speed up development.

🎥 Last but not least, we have a short bytesize talk showing the necessary steps:

Play

Additional bug fixes and improvements

  • Have you seen a no space left on device error or your CI tests lately? This is now fixed by adding a clean up step in the github actions.

  • New api-docs on the nf-core website to get clarifications on linting errors etc.

  • Updating a module won’t remove custom nextflow.config files anymore.

  • The commands nf-core modules test and nf-core subworkflows test now have a new parameter --profile to specify the container to run nf-test with.

Thanks to all the contributors who made this release possible, especially to Harshil Patel and his team for kicking off the lib restructuring and the utils_* subworkflows.

You can find the whole changelog on GitHub.


Published on
19 February 2024