[Bioc-devel] About the reproducibility of BioC server environment

[露崎 弘毅]

Dear Bioc Core Team

I have created many Bioconductor packages, but it is always hard to pass the R CMD CHECK/BiocCheck.

Especially, it is still difficult to construct the identical environment of single/daily package builder in the local machine.

Sometimes the error is caused by the difference of the OS (e.g. Windows ↔︎ Linux, Max), but it is very difficult to trace the reason in the local machine.

In the "daily" package builder case, the code can be tested only one time per day, and it will slow down the package development speed.

I have often experienced that the BiocCheck in tokay1 (Windows) is failed due to the CheckTime, but it is difficult to distinguish whether the error is caused by my source-code or the machine is just busy for the task of other packages.


Why don't you create the docker containers for the single/daily package builder?
https://www.bioconductor.org/help/docker/

I think that it will be very helpful for many BioC developers.

I want to use the system like below.


docker run bioconductor/single_windows
docker run bioconductor/single_linux
docker run bioconductor/single_mac
docker run bioconductor/daily_windows
docker run bioconductor/daily_linux
docker run bioconductor/daily_mac


Koki Tsuyuzaki

[[alternative HTML version deleted]]

[Martin Morgan]

Thanks for the comments.

A docker image doesn't help with differences between operating systems.

A docker image wouldn't be practical for the build system, where there are 1000's of packages installed.

A docker image would need to be updated daily, since that is what the build system does; you would need to pull the docker image.

A docker image with appropriate R and base packages is available; see https://bioconductor.org/help/docker/ but note that these require that your own dependencies be installed and kept current.

Windows builds and tests on 32 and 64 bit platforms, so can take a lot more time; it is difficult to differentiate between this sort of problem and timeouts caused by OS-specific or package issues.

Usually avoidable problems come about because the wrong version of Bioconductor is in use, or because packages are not current. These can be checked with

BiocManager::version() == "3.9" # alternate: BiocManager:::isDevel()
BiocManager::valid()

In the big picture I hope that we can arrive at a more continuous version of the builders, but that will not happen soon.

Martin

On 11/6/18, 8:37 AM, "Bioc-devel on behalf of 露崎 弘毅" <bioc-devel-***@r-project.org on behalf of k.t.the-***@hotmail.co.jp> wrote:

Dear Bioc Core Team

I have created many Bioconductor packages, but it is always hard to pass the R CMD CHECK/BiocCheck.

Especially, it is still difficult to construct the identical environment of single/daily package builder in the local machine.

Sometimes the error is caused by the difference of the OS (e.g. Windows ↔︎ Linux, Max), but it is very difficult to trace the reason in the local machine.

In the "daily" package builder case, the code can be tested only one time per day, and it will slow down the package development speed.

I have often experienced that the BiocCheck in tokay1 (Windows) is failed due to the CheckTime, but it is difficult to distinguish whether the error is caused by my source-code or the machine is just busy for the task of other packages.


Why don't you create the docker containers for the single/daily package builder?
https://www.bioconductor.org/help/docker/

I think that it will be very helpful for many BioC developers.

I want to use the system like below.


docker run bioconductor/single_windows
docker run bioconductor/single_linux
docker run bioconductor/single_mac
docker run bioconductor/daily_windows
docker run bioconductor/daily_linux
docker run bioconductor/daily_mac


Koki Tsuyuzaki

[[alternative HTML version deleted]]

_______________________________________________
Bioc-***@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

[Kasper Daniel Hansen]

In my experience, if I aggressively update my development installation of
Bioconductor (including occasional updates of R-devel when appropriate) I
can usually replicate most-all failures on the build server which has the
same OS as my local machine. It does require aggressive updating though.

Post by Martin Morgan
Thanks for the comments.
A docker image doesn't help with differences between operating systems.
A docker image wouldn't be practical for the build system, where there are
1000's of packages installed.
A docker image would need to be updated daily, since that is what the
build system does; you would need to pull the docker image.
A docker image with appropriate R and base packages is available; see
https://bioconductor.org/help/docker/ but note that these require that
your own dependencies be installed and kept current.
Windows builds and tests on 32 and 64 bit platforms, so can take a lot
more time; it is difficult to differentiate between this sort of problem
and timeouts caused by OS-specific or package issues.
Usually avoidable problems come about because the wrong version of
Bioconductor is in use, or because packages are not current. These can be
checked with
BiocManager::version() == "3.9" # alternate: BiocManager:::isDevel()
BiocManager::valid()
In the big picture I hope that we can arrive at a more continuous version
of the builders, but that will not happen soon.
Martin
On 11/6/18, 8:37 AM, "Bioc-devel on behalf of 露崎 弘毅" <
Dear Bioc Core Team
I have created many Bioconductor packages, but it is always hard to
pass the R CMD CHECK/BiocCheck.
Especially, it is still difficult to construct the identical
environment of single/daily package builder in the local machine.
Sometimes the error is caused by the difference of the OS (e.g.
Windows ↔︎ Linux, Max), but it is very difficult to trace the reason in the
local machine.
In the "daily" package builder case, the code can be tested only one
time per day, and it will slow down the package development speed.
I have often experienced that the BiocCheck in tokay1 (Windows) is
failed due to the CheckTime, but it is difficult to distinguish whether the
error is caused by my source-code or the machine is just busy for the task
of other packages.
Why don't you create the docker containers for the single/daily package builder?
https://www.bioconductor.org/help/docker/
I think that it will be very helpful for many BioC developers.
I want to use the system like below.
docker run bioconductor/single_windows
docker run bioconductor/single_linux
docker run bioconductor/single_mac
docker run bioconductor/daily_windows
docker run bioconductor/daily_linux
docker run bioconductor/daily_mac
Koki Tsuyuzaki
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/bioc-devel
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/bioc-devel

[[alternative HTML version deleted]]