Maintaining BOINC: BOINC

Showing posts with label BOINC. Show all posts

BOINC Development Status Report: October 2024

This month was relatively calm, with fewer events.

David and I are working on adding Docker support. Most of the core work is done, but now we’re refining several parts of the existing functionality within the BOINC client to ensure compatibility with the new type of applications. A major advantage of Docker support, combined with WSL, is that it simplifies the infrastructure significantly. This makes things easier for project developers, as they no longer need to create separate applications for different operating systems. Although Android will still require a dedicated application, having a unified solution for desktop OSs is a significant step forward for BOINC. This also means projects can potentially drop VirtualBox support in favor of Docker, which is simpler and more flexible.

For more details about Docker and WSL, please refer to the following resources:

With Docker support, we can also run applications that were previously unsupported, which is very important for expanding BOINC’s functionality.

This year, I attended the GCB conference in Bielefeld, Germany, where I spoke with several scientists. While they were generally interested in BOINC, their first question was whether it could run Docker containers—a crucial capability for them. Most of their applications are written in Python and require additional packages to be installed on the system. Given BOINC's current architecture, distributing workloads across hosts is challenging. Additionally, many potential users interested in BOINC lack the time and expertise to set up their own project servers or configure applications correctly.

Our responsibility as BOINC maintainers is to simplify this process as much as possible and remove as many barriers as we can. Docker can be a significant help here, as it allows users to configure their environment once, which BOINC can then replicate automatically. While a project server is still required, we aim to use BOINC Central for smaller research projects, providing access for scientists who may not have the resources or skills to create their own server.

BOINC Workshop 2024: What's next?

This year, the BOINC Workshop was hosted by CERN in Geneva. I was very excited to go there and finally meet the people I had been working with virtually for more than five years. I had planned to write a blog post about this at the beginning of June, right after the Workshop. But after spending three days there, I decided to take some time to reflect on the experience. Over the last four months (and even longer by now), I’ve tried to write this post, but each time I wasn’t ready. However, there’s no reason to hold back any longer, and I feel it's time to share my thoughts.

I don’t want to repeat what David Anderson has already written, so I’ll just link to his post here: https://continuum-hypothesis.com/trips/geneva_24.php. It’s a very detailed and good report about the BOINC Workshop, the WSIS prize (and the cake!), and Geneva in general. My thoughts about those three days are pretty much in line with David’s.

The people were great, and the location was nice, but I left with a slightly bitter taste. Mostly because I was expecting to see more new faces - people who are new to BOINC - but instead, I only saw familiar ones.

I fully understand that the world has changed, and people are excited about entirely different things now.

I remember myself about 15 years ago, staring at the graphics of the SETI@Home application. It was completely incomprehensible to me, but still mesmerizing. My PC back then had a single-core 1.8 GHz CPU, and I saw that finishing just one SETI@Home work unit would take several hours, yet I had the feeling that my computer was doing something great and important.

Today’s youth get bored if something takes longer than 10 seconds (is that TikTok’s fault?). I’m not that old (almost 35), but I see that things that used to excite me and the people I knew just don’t capture interest anymore. And I don’t like that.

I don’t think I can change this on a global scale, but I still believe in BOINC and its mission. I still believe BOINC is important. I know there are a lot of smart people with great ideas, but they often lack the means to implement them. Our mission is to help them.

During the Workshop, we discussed the need to implement Docker support in BOINC since it could significantly accelerate research. Scientists today no longer create applications in C/C++ or similar languages — they prefer using Python (which is slower to run but faster to develop). Having the ability to run complex Python scripts on BOINC without having to worry about the different platforms those scripts might run on is incredibly important. David Anderson is working on the first draft of a Docker wrapper for BOINC, and when it’s released, I’m sure it will open a new chapter in BOINC’s story of success.

BOINC Release 8.0.0 and liblzma vulnerability

Recently it was discovered that liblzma - the popular library for data compression/decompression - contains a severe vulnerability: https://tukaani.org/xz-backdoor/

Unfortunately, BOINC 8.0.0, that is available for alpha testing, was built with the malicious 5.6.0 version of this library.

This is because liblzma is the dependency of libtiff that is a dependency of wxWidgets, that is used to build the GUI of BOINC Manager.

As far as we can see, this issue doesn't affect our users, since the target of this 'backdoor' is an sshd process, and BOINC doesn't work with it in any way. Just to highlight one more time: only BOINC Manager was built with this library, and BOINC client doesn't use this dependency at all. So we strongly believe that our users are completely safe while using BOINC version 8.0.0

But since the analysis of this backdoor is not finished yet, we have decided to build another release 8.0.1 that downgrade the liblzma version to 5.4.4 that is completely safe. According to the available data, this backdoor works on Linux only, that is why BOINC 8.0.1 will be released for alpha testing for Linux only.

We will continue monitoring the situation, and if needed, will release 8.0.1 for other platforms as well.

Please use this page to find the instructions about using the BOINC Linux packages: https://boinc.berkeley.edu/linux_install.php

Thank you for understanding.

Stay safe!

Major BOINC version change

One may wonder, looking a the list below, why to change the major release version if there are not so much changes made after the last release.

The main reason for that is the support of the new type of application that allows running AI/ML applications on BOINC infrastructure, that have different requirements. It's explained in more details here: https://github.com/BOINC/boinc/wiki/Sporadic-Applications

Also, starting from the upcoming release, we will provide our own BOINC vanilla packages for the major Linux distributions: https://aenbleidd.blogspot.com/2024/02/vanilla-boinc-packages-reason-and.html

Although the current list of supported Linux distributions is quite short and contains only 4 of them, more will be added later. We tried to concentrate on the most popular distributions, and will add others step-by-step.

In this release the detection of Android GPUs was finally fixed: https://aenbleidd.blogspot.com/2023/08/android-boinc-where-are-my-gpus.html

This Release Candidate should be available for a public alpha testing before the end of March 2024.

The list of the most important changes, introduced in the next release you can find below.

BOINC 8.0.0 Release Candidate

New features

All (Android, Linux, MacOS, Windows)

Add support of Sporadic Applications

Android

Linux

Building DEB (Debian 10/11/12, Ubuntu 20.04/22.04) and RPM (Fedora 37/38/39, openSUSE 15.4/15.5) packages

MacOS

Add 'Apple GPU' as an official GPU type

Bug fixes

BOINC Desktop (Linux, MacOS, Windows)

Android

Linux

MacOS

Windows

Android BOINC: where are my GPUs?

Once we made a new release of Android BOINC several years ago, and immediately after received several similar bug reports of the missing GPU detection. That was a very interesting case, because between the releases we made no changes to the GPU detection, and still on the same Android OS, old version was working fine, and the new one not.

Since there was no any Project that has an application for any Android GPU, this issue was classified as non-blocker, and we didn't rollback that release.

Still this case was very interesting, and I have started the investigation of it. And at that moment I started to realize, that every new Android release breaks something that was working for years. Of course, main explanation of such an actions was a security issues.

From the very beginning, Android was a very open platform (in comparison to the iOS for example), and it allowed you to do almost anything you want. But, of course, this leads to security issues, including the major ones.

Android BOINC application is very different from the Desktop implementation. First of all, BOINC Manager is written on Java and Kotlin (originally it was written on Java, but we have started to migrate it to Kotlin as it is more modern, safer and easier to maintain). But BOINC client is still written on C++, because reimplement all the functionality of it on Java/Kotlin would be too much work, and not everything required by the BOINC client was yet available (and some stuff still not) in the Android API.

But Android API from version to version started to deprecate and even remove the functionality that is crucial for running BOINC on Android. For example, we are not able to upgrade to the latest version of the Android API because then we will not be able neither to run the client natively, nor to run the downloaded Projects' applications. We spent quite a lot of time trying to understand how to proceed with this, and at that moment decided that we will stick to the current version of Android API as long as possible. But as a drawback, we are not able to publish Android BOINC on the Play Store, because Play Store policies demand to use the newer version of the Android API that doesn't work for us. That is why currently Android BOINC is published on the official BOINC website and on F-Droid (3rd party service not supported by us) only.

Another issue was with the GPUs. In this case there was a change in the API that prevented BOINC from listing available GPUs on the device. Luckily, there still was a way to implement this GPU discovery in a little bit weird way, but at least it works.

However, there is still no any Project that has the application for Android GPU, but we really hope that with the fix we provided, once we will see such a Project that will allow BOINC to utilize Android device more effectively than now.

Vanilla BOINC packages: the reason and the purpose

I have very controversial feelings about open-source software and Linux OS. From one hand, I like the idea of the transparency, when you can always check what is really doing every application you run. From the other hand, as a maintainer of the application, you lose some part of control on it, because it is possible to modify some application's behavior (e.g. via distro patches), that contradicts the original idea of it or break some functionality. And because there are so many different OSs, you can't keep an eye on all of them. That is why sometimes you receive a bug report that looks completely weird, and when you start doing the investigation, you realize, that this happened because of the distro patch, that was intended to fix another issue, but as a result introduced a new one. And, as a maintainer of the application, there are not so much options for you here. But in any case, you have unsatisfied user, that is always bad.

We, at BOINC, faced several times the situation when the package that is distributed with the distro is completely broken, and users have no idea how to fix that. You can post instructions, blog posts, answer questions of forums, but still the same question about the same bug will be asked. Sometimes the communication with the distro maintainers becomes hot, and every party is completely sure that only their opinion is correct.

To get rid of this situation, we decided to create our own BOINC vanilla builds for Linux. We have started with the most popular distributions: Debian, Ubuntu, Fedora and openSUSE. Currently we target only the LTS releases that are officially supported, so that is why for now we provide packages for Ubuntu 20.04 and 22.04 only.

More details (including the list of supported OSs) are here: https://github.com/BOINC/boinc/wiki/Linux-DEB-and-RPM-support

Since building all these packages (20 in total for every version) is quite a task, I had to try to automate this as much as possible, including adding some integration tests to avoid very dumb bugs. I can't say that the current CI is finished, but it does its work already: every night we have a new build that is automatically published to the 'nightly' channel on the BOINC official website. This gives our users the opportunity to test new features as soon as they added to the codebase even before the alpha testing phase begins. I can't say this is a recommended way of installing BOINC, but since we have now 2-3 releases every year, it might be helpful for those who have problems with the issues that are blockers for them only.

These packages are not yet recommended to use, but the first result is already very promising. On the video below the process of installation of both BOINC client and Manager on Ubuntu 22.04 is shown:

Of course, there are still some issues, but now it will be much easier to fix them and deliver to our users faster.

Stay tuned for further updates.

Introduction

Several weeks ago I was working on a solution for a bug that was in Android BOINC for at least 4 years. This bug was very minor, that is why it was not fixed during all this time.

This was not the easiest bug to fix, and it required quite a deep research into Android internals, so I decided that it would be nice to share my findings (there will be a post about it later).

Usually I don't like writing long posts, but sometimes it's very important to share some information, including the technical one, with the people who might be interested in it. Also, such a posts might bring some light to the technical and design solutions in BOINC.

So, if any of you ever asked a question like "Why this was done in BOINC in this way?", you could probably find some answers in this blog.

I'm not a blogger, but I'll try to share more information about BOINC and what happens around it. This blog will not contain any personal and/or any other information that is not related to BOINC in one or another way. Also, the posts will be more or less big. That is why it will be updated very rare. Shorter but more frequent news and updates I usually post on my X/Twitter and/or Mastodon pages.