2022Q3 Roadmap

Linux packaging

Breifcase's AppImage packaging backend has had a cluster of issues reported related to inclusion of third party libraries, and the post-processing that needs to be performed on an AppImage. We've already started work on this during Q2; early in Q3, you should expect to see this work wrapped up. This will include:

• Adding support for linuxdeploy plugins for our existing AppImage backend; and
• Adding a Flatpak backend.

Windows packaging

Briefcase has an MSI packaging backend for Windows; and while this backend works, it doesn't deliver an end-user experience that is as polished as we'd like. We should be starting initial work on Windows packaging in the coming week, with the complete solution completed in early Q3. This will include:

• Adding a stub binary for Windows apps, so that Briefcase apps present to the task manager with the apps name and icon, rather than "pythonw";
• Improving log capture on Windows, so that it is easier to diagnose when problems occur; and
• Fixing some issues with the way the site is defined and used. This may also address some issues we've seen with binary modules in some Windows apps.

Binary modules

The biggest caveat on BeeWare's mobile app story has always been that we don't provide support for binary packages on mobile platforms. In Q3, we're going to start addressing this limitation.

There are already tools like crossenv that can simplify this process, and we've got a PR in our review queue that provides a proof-of-concept implementation of Android binary packaging. We need to take these tools, plus other tools in the Python ecosystem, and extend Briefcase so that we can present a clear development story for mobile developers that want to use modules with binary components.

On iOS, there is an additional complication - we need to resolve how to manage dynamic loading of modules. BeeWare's iOS support currently relies on static compilation of all binary modules. This was originally done because prior to iOS8, dynamic linking was prohibited on iOS. The days of iOS8 are now long behind us; but the Apple support package has retained this old approach, mostly because it wasn't clear what approach Apple would allow through the App Store review process. We will be making changes to Briefcase's iOS support to accomodate dynamic module loading, and then extending that support to include user-provided binary modules.

Testing (especially GUI testing)

Without testing, it's very difficult to ensure the stability and reliability of code in the long term. Unfortunately, many of the areas where BeeWare operates are inherently difficult to test. Visual GUI tests are notoriously difficult to perform; packaging tests require the ability to actually install and run finished applications; testing mobile applications requires access to mobile device hardware in a CI setup. As a project, we've avoided writing tests for many of our core features due to these complexities - and we've paid the price for this in regressions. Now that we have more time, we can focus on setting up the frameworks to let us perform this testing.

This includes adding:

• Support package tests, ensuring that the packages include valid libraries;
• App template tests, ensuring that Briefcase is able to generate valid and installable applications; and
• GUI tests, ensuring that Toga generates apps that look and behave the way they should.

Testing is an ongoing process, so it's highly unlikely we'll have "100% test coverage" by the end of the quarter. The goal will be to add the frameworks that allow us to add tests at all, so that we can retrofit tests onto existing code, and add tests on all new code going forward. Ideally, these tests will be fully automated tests running in CI. However, we may accept semi-automated or manual tests as a short term of interim measure, especially if those tests lay the groundwork for fully automated tests in the future.

Conda integration

Briefcase is currently uses PyPI and pip as source of Python packages. However, PyPI and pip isn't the only packaging solution in the Python ecosystem - another commonly used packaging source is Conda.

At this point, I must recognise the elephant in the room - Anaconda, the company that produced and maintains Conda, is my employer. However, I must stress that the fact this is on our roadmap is not as a result of a directive from my employer. The original proposal for this idea predates even my earliest hiring discussions with Anaconda. It is being pursued now for three reasons:

1. The key value of Conda is that it simplifies the process of delivering complex binary packages, and ensuring that the binary dependencies for those packages are isolated from any packages that might be present on the user's computer. This core problem statement shares a lot of similarities with the core problem associated with application packaging. Conda also has much more nuanced representation of hardware platforms, alternatives for binary artefacts, and options for custom repositories. There is a possibility that using Conda as a packaging base may simplify some of the problems we've seen with distributing binary dependencies.
2. There is a significant portion of the Python community that have commercial needs that requires the use of the managed repositories provided by Anaconda, or are simply more familiar with Conda tooling than PyPI/pip.
3. As a result of having 1 (soon to be 2) full time engineers, we now have the resources to explore complex changes that might not have otherwise been explored. We are exploring this idea because we now have the resources to do so; we were not provided those resources on the condition that we explore this idea.

We don't anticipate dropping support for pip, or for PyPI as a package source. We're looking at extending Briefcase to provide another commonly used and requested option for packaging, rather than replacing PyPI/pip.

Upstreaming

The Apple and Android support packages maintained by BeeWare include a number of patches against the core CPython. We want to begin the process of getting these modifications into the CPython codebase.

There have been two major impediments to this in the past:

1. Development resources When we were a volunteer driven project, we didn't have the resources to engage meaningfully with the CPython core team to shepherd our patches into the CPython codebase.
2. Testing Supported platforms in the CPython codebase need to have CI solutions.

We now have resources, so we can dedicate the time needed to upstream patches; and once we've completed our testing work in Q3, we'll be in a position to recommend CI configurations to the CPython core team. That means the work of upstreaming patches and making iOS and Android fully supported platforms can begin.

Toga 1.0

The broad goal for "Toga 1.0" could be considered "Tkinter, but with some additional bells to show off". We want to deliver a core widget set of the "essentials", plus some nice features to show off, like WebViews and camera or GPS integration. The exact set of features for Toga 1.0 is still a subject for discussion; I'd expect to see a more concrete feature list and plan emerge in Q4.