2023Q2 Roadmap

Posted by Russell Keith-Magee on 4 April 2023

It doesn't feel like it's been three months since our last roadmap update... but here we are, in April 2023. As always, this roadmap should be read as a guide to what we aim to focus on over the coming quarter, rather than a hard commitment of features that will be made available on a specific deadline.

Q1 progress

Our focus this quarter has been on testing; however as with our last report, our progress has not been as significant as we hoped it would be. Getting the first tests in place has proved to be much more complex than we anticipated. In order to test widgets at all, we've had to set up all the infrastructure that will be necessary to test every other widget, such as defining the probes to inspect colors and fonts, and to ensure that the GUI framework has finished applying changes before performing a test assertion. The process of building the first tests has exposed a number of problems deeply embedded in the core of Toga - problems that either aren't observable or are easily masked when you're writing an application as a user, but can't be avoided when you're writing a test suite that will run fast and needs to have repeatable behavior. We're also testing the limits of the features exposed by each platform - and everything we do needs to be repeated on 5 GUI toolkits with 5 completely different views of the world.

The good news is that as of the end of Q1, we're finally getting traction on the problem. We're about a quarter of the way through the widgets by count. Admittedly, we've been tackling the simpler widgets first - but the lessons learned (and bugs fixed) making those simple widgets work should make testing the more complex widgets easier. There's also some commonality in the more complex widgets - for example, TextInput, PasswordInput and NumberInput are all essentially the same widget, so once tests are built for one of them, the other 2 should fall out reasonably easily.

We have also been able to release Toga v0.3.0. This release was almost 5 years in the making.

Q1 saw some unexpected improvements to Briefcase. We've had persistent problems supporting AppImage and Flatpak as packaging formats; during Q1, we couldn't avoid these problems any longer, so we introduced a System packaging backend, and made this the new default for Linux apps. We're now able to produce native system packages for Briefcase apps, supporting almost every Debian, Red Hat or Arch derived distribution. These packages are already proving more stable: we had no major issues during a recent tutorial at Everything Open 2023, despite a wide variety of Linux distributions in the room - something that we've never been able to say about AppImage.

However, once again, that means there's a lot left on the Q1 TODO list. Toga documentation is being slowly updated as we audit each widget; however, we haven't made any changes to the example apps; we haven't been able to do any substantial Toga planning; and we haven't been able to add any native cross-platform hardware support.

Q2 priorities

Our short term goals are, therefore, largely unchanged from last quarter:

  • Testing - Completing the Toga GUI testing suite will again be our primary focus for the quarter.
  • Planning - Once we've got the test harness in place, we'll be in a much better position to evaluate what work remains to get to v1.0.
  • Documentation - The process of systematically building a test suite also gives us an opportunity to update the API documentation for each widget as we go. There's still lots of documentation required on top of the API guide, but complete API documentation will be a big start.
  • Cross-platform hardware - Cross-platform support for device hardware (especially mobile) is one of the banner features that makes mobile device support worthwhile. Providing proof that this hardware support is easy to use will be a key demonstrator for BeeWare as a platform.

Longer term goals

Our longer term goals are also largely unchanged. Upstreaming, Toga 1.0, and a forge for mobile packages are all important goals for the BeeWare ecosystem. However, we don't expect to make any significant progress on most of these goals until we have solidified Toga's testing story. We should, however, see at least the first signs of progress on upstreaming BeeWare's work on mobile as a result of PyCon US, where we'll have a chance to raise issues directly with the Python core team.

Summary

Providing a robust GUI testing framework for Toga has definitely proven to be a lot more complicated than we originally thought. The good news is that as of the end of Q1 2023, we're finally starting to make good progress on testing. We hope that by the time of our next status update, we're able to report that we're nearing the end of our testing journey, and we can turn our focus to other areas of interest.

March 2023 Status Update

Posted by Russell Keith-Magee on 3 April 2023

March has been a month of slow but steady progress for BeeWare.

What we've done

  • Progress on GUI testing in Toga is continuing. We now have 7 widgets fully tested, with another 2 in progress. As with last month's update, getting these tests passing has required wrestling a lot of fundamentals, in particular around the simulation of events such as mouse clicks. We're also discovering (and fixing) lots of corner cases in widget implementations that only surface when you're doing rigorous and repeatable testing.
  • We've made some major changes to the way GTK widgets are rendered. This change significantly reduces the amount of re-rendering that is performed by GTK, as well as resolving a number of long standing issues related to resizing widgets.
  • Thanks to some community contributions, we were were able to add support for focus events and text alignment on GTK's TextInput widget, and support for text alignment and fonts on GTK's NumberInput widget.
  • We landed the implementation of a Linux system package backend for Briefcase, which can output DEB and RPM packages. Thanks to a community contribution, we were also able to add support for Arch packages
  • We modified the locations where Briefcase outputs build artefacts. This removes some sources of bugs associated with tools not supporting spaces in filenames, and makes Briefcase's output consistent with other tools in the Python ecosystem.
  • We made significant progress on modifying Briefcase's AppImage and Flatpak backends to use Indygreg's Standalone Python, rather than BeeWare's Linux support package. Standalone Python is becoming a de facto standard for pre-compiled Python builds (especially on Linux), so it makes sense for BeeWare to converge on this common standard. This change will also significantly speed up Flatpak builds, should be more stable on AppImage, allows us to switch to manylinux base images - and it reduces the number of projects that BeeWare is responsible for maintaining. We're currently waiting on a fix to the linuxdeploy GTK plugin to be merged before we can land this change.
  • Thanks to a community contribution, we added support for PyGame to Briefcase.
  • We migrated BeeWare's CI to use the new System packaging backend on Linux. This means that CI is significantly faster, as building system packages requires a lot less processing than an AppImage.
  • We released Travertino 0.2.0. Travertino is the library that underpins Toga's Pack layout algorithm. It's been several years since we last did a Travertino release, but some recent bug fixes warranted an update.
  • We presented a tutorial at Everything Open 2023.

We also made a big change to our social media strategy: we started a Mastodon account. We've wound down our Twitter usage at the end of last year; going forward, you can get your bite size BeeWare updates from @beeware@fosstodon.org.

What's next?

In terms of technical goals, April will continue our push for GUI test coverage. However, we're expecting progress to be a little slower, because of one big event - PyCon US. We'll have a booth in the community section of the conference floor; the core team are presenting 2 talks; we're presenting at the Language Summit; and we'll be at the sprints for the first 3 days. Preparing for these talks will take up a lot of our time in April, as will the event itself - so progress is likely to be a little slower than normal. If you're going to be there, come by the booth and say hi! If you'd like to help us to staff the booth - get in touch on Discord, on Mastodon, or email!

Want to get involved?

Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:

  1. Modify the handling of app signing options
  2. Filter out a message generated after Xcode updates
  3. Add the ability to refresh the support package in a project
  4. Provide protection against creating projects with "semi-reserved" names
  5. Add the ability to configure the ABIs built by an Android project
  6. Add support for the ANDROID_HOME environment variable
  7. Improve validation of project names in the Briefcase wizard
  8. Rationalise the application of adhoc signing on macOS
  9. Correct the capture of log messages on the web backend

Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.

February 2023 Status Update

Posted by Russell Keith-Magee on 1 March 2023

Although there haven't been a lot of new BeeWare features developed in February, the work that has been completed represents some important progress against our overall project goals.

What we've done

  • The biggest milestone is that we have achieved 100% test coverage for our first widget - Button! This may sound a little underwhelming - after all, Toga has had a Button widget for a long time, and has been fairly reliable. However, getting to this point has required a lot of smaller (seemingly unrelated) problems to be solved. It has also required setting up all the infrastructure that will be necessary to test every other widget, such as defining the probes to inspect colors and fonts, and to ensure that the GUI framework has finished applying changes before performing a test assertion). We've also taken this opportunity to start migrating the Toga unit tests to pytest, which promises to significantly improve our testing capabilities.
  • We formalised the relationship between Toga's Pack layout model and CSS. This should make it a lot easier to determine what "correct" layout behavior should be.
  • We identified a low level issue with the way that layout is performed on GTK, and possibly with other platforms. We're still working through the consequences of this change, but it has indirectly resolved a number of issues we have seen reported with widget layouts, especially on GTK.
  • We've added the ability to pass command line arguments to running apps from the briefcase dev and briefcase run interface. This was primarily done to make it easier to run unit test suites; but it can be used for any other reason you might want to use a command line argument.
  • We have a draft implementation of a Linux system packaging backend for Briefcase. We've had consistent problems with both AppImage and Flatpak, most of which can be traced back to resolving links to system packages. We're not going to remove these backends - when AppImage and Flatpak work, they're great options - but we're going to change the default packaging backend on Linux to be system packages (i.e., DEB/RPM etc). This makes the simple case of building a package for your own system much easier, while retaining the ability to build packages for other distributions as required. We're hoping to land this PR in the coming month.
  • We corrected an issue with threading modes that prevented some Windows apps (especially PySide/Qt apps) from displaying dialogs.
  • We corrected an issue with the Briefcase Web backend that would cause intermittent errors when loading a web app.
  • We updated our CI infrastructure to make better use of shared task definitions. BeeWare has a lot of Github repositories, and a lot of common tasks (such as rolling out an app across multiple Python and operating system versions). Previously, each Github repository had an independent set of CI task definitions; we've now got a central collection of task definitions that is used by all repositories.
  • We cleaned up the table in Toga's documentation describing the state of support of each widget. This allows us to differentiate between widgets that we've fully tested, and widgets that exist, but haven't been tested.
  • Thanks to a community contribution, we've been able to publish a backlog of iOS binary packages for Python 3.8-3.11. For the packages we currently support (including Numpy, Pandas, Pillow and more), we're now able to offer an iOS binary wheel for every published version for which there is a corresponding macOS wheel, going back to 2019.

What's next?

In March, we'll be continuing to improve GUI test coverage. Now that we've got one widget to 100% coverage, we know we have all the core testing infrastructure in place, so it should be much easier to implement tests for the future widgets. By the end of March, we should have a lot more widgets at 100% coverage.

We'll also be presenting at Everything Open 2023. If you're attending, come say hi!

Want to get involved?

Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:

  1. Modify the handling of app signing options
  2. Filter out a message generated after Xcode updates
  3. Add the ability to refresh the support package in a project
  4. Provide protection against creating projects with "semi-reserved" names
  5. Add the ability to configure the ABIs built by an Android project
  6. Add support for the ANDROID_HOME environment variable
  7. Add a template branch option when creating a new project
  8. Improve validation of project names in the Briefcase wizard
  9. Rationalise the application of adhoc signing on macOS
  10. Correct the capture of log messages on the web backend

Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.

January 2023 Status Update

Posted by Russell Keith-Magee on 1 February 2023

BeeWare has had an exciting start to the new year, with a couple of significant releases

What we've done

We had two big releases in January.

Firstly, we released Briefcase 0.3.12. This is the first Briefcase release in 3 months, so it includes a lot of improvements, but the most notable improvements are the new "Testing mode", a major improvement to the Android build process, and a huge number of bug fixes and improvements - especially on Windows.

Secondly - and more importantly - we released Toga 0.3.0! No more dev releases or --pre flags for pip - the stable release of Toga on PyPI is now closely tracking actual development. It's been over 5 years since we last released an official stable Toga version, even though we've been recommending the dev releases for most of that time. However, as a result of a number of changes that landed this month, we're finally committing to the 0.3.0 branch properly.

These releases incorporate changes going back months (and years); but we made some very specific progress during January:

What's next?

Much of the last 3 months have been spent getting a GUI testing framework in place for Toga. That work is now in place, so now we'll be working full time on writing GUI tests for Toga, with the aim of getting Toga to 100% coverage. Along the way, we'll discover all the edge cases, bugs, and partially missing implementations that are lurking in the individual Toga widgets.

Want to get involved?

Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:

  1. Modify the handling of app signing options
  2. Filter out a message generated after Xcode updates
  3. Add the ability to refresh the support package in a project
  4. Provide protection against creating projects with "semi-reserved" names
  5. Add the ability to configure the ABIs built by an Android project
  6. Add support for the ANDROID_HOME environment variable
  7. Purge cache folders when installing app code in Briefcase
  8. Add a template branch option when creating a new project
  9. Improve validation of project names in the Briefcase wizard

Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.

2023Q1 Roadmap

Posted by Russell Keith-Magee on 11 January 2023

Welcome to a new year! 2022 saw some amazing progress for BeeWare; and we've got some big plans for 2023. As always, this roadmap should be read as a guide to what we aim to focus on over the coming quarter, rather than a hard commitment of features that will be made available on a specific deadline.

Q4 progress

Progress in Q4 was not as significant as we originally planned. A combination of conference travel and holiday leave meant the quarter was shorter than most; but even then, our goals for the quarter were extremely optimistic.

We were able to hit our goal for Web deployment. We've restored the Toga web backend to a working state, and we've added a Web deployment backend to Briefcase. There's still a lot of work required before Web deployment is a viable as a production solution, but the initial pieces are now in place.

We were also able to hit our goals on housekeeping. We've made a lot of improvements to the base infrastructure of repositories, enabling Black, pre-commit, and towncrier in Toga, and improving coverage reporting (including getting to 100% branch coverage in Briefcase's test suite).

We've made significant progress on improving the testing story for BeeWare apps. We've added the ability for Briefcase projects to run normal Python test suites inside a packaged app, and we've demonstrated that we can use this capability to run CI for iOS and Android apps (as well as desktop platforms). This feature will be included in the next Briefcase release.

Using this new testing capability, we've started building a GUI test suite for Toga. This is a critical part of ensuring the quality of Toga going forward; but getting this testing framework in place has revealed lots of complications and bugs. We're making progress, but there's still a lot of work to be done.

Our plans to release Toga v0.3 final have also been delayed. The process of adding SCM-based versioning to the repository (part of the housekeeping task) revealed a bug that prevented code running on Android; fixing that bug became a blocker on the release of Toga v0.3. We've now resolved that problem, so we should be able to release Toga v0.3 final as soon as the fix has been released into the BeeWare's Android tooling.

Unfortunately, that leaves a lot on the Q4 TODO list. We haven't made any changes to the example apps; Toga documentation is mostly unchanged; we haven't been able to do any substantial Toga planning; and we haven't been able to add any native cross-platform hardware support.

Q1 priorities

It's probably unsurprising that Q1 2023 will be picking up on items we didn't get around to in Q4 2022.

  • Testing - Completing the Toga GUI testing suite will be our primary focus for the quarter.
  • Planning - Once we've got the test harness in place, we'll be in a much better position to evaluate what work remains to get to v1.0.
  • Documentation - The process of systematically building a test suite also gives us an opportunity to update the API documentation for each widget as we go. There's still lots of documentation required on top of the API guide, but complete API documentation will be a big start.
  • Cross-platform hardware - Cross-platform support for device hardware (especially mobile) is one of the banner features that makes mobile device support worthwhile. Providing proof that this hardware support is easy to use will be a key demonstrator for BeeWare as a platform.

Longer term goals

Our longer term goals are largely unchanged from Q4 2022. Upstreaming, Toga 1.0, and a forge for mobile packages are all important goals for the BeeWare ecosystem. However, we don't expect to make any significant progress on these goals until we have solidified Toga's testing story.

Summary

The seeming lack of progress made in Q4 was definitely disappointing; but the work hasn't been wasted. Progress has been slow specifically because the problems we're trying to solve are hard, and reveal every gnarled edge case that platforms have to offer. We are making progress, though; and once we've got the Toga testing framework stabilized, we'll be in a much stronger position to make predictions for future work on Toga.

December 2022 Status Update

Posted by Russell Keith-Magee on 17 December 2022

This status update is a little earlier than our usual end-of-month report because the team will be taking a well-earned break to spend time with family and friends over the holiday and new year period. As a result, we've got less to report than in past months; however, some significant progress and improvements have been made.

What we've done

During December:

What's next?

There won't be much more progress from the core team for the rest of this year. We'll still be around to handle critical problems, answer questions and do code reviews for contributors; but our response times might be a little slower than normal. We'll publish our Q1 2023 roadmap when we return in January - but we'll be largely picking up where this year has left off - improving the testing story for Toga.

Want to get involved?

Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:

  1. Modify the handling of app signing options
  2. Filter out a message generated after Xcode updates
  3. Add the ability to refresh the support package in a project
  4. Provide protection against creating projects with "semi-reserved" names
  5. Add the ability to configure the ABIs built by an Android project
  6. Add support for the ANDROID_HOME environment variable
  7. Purge cache folders when installing app code in Briefcase

Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.

November 2022 Status Update

Posted by Russell Keith-Magee on 1 December 2022

November has brought some very important improvements to BeeWare. Although these changes don't have an immediate impact for users, they're going to form a vital part of our work going forward.

What we've done

During November:

What's next?

December will be a slow month due to the Christmas and New Year break; however, until then, we'll continue to focus will continue to be on testing. We'll continue to build out the testing tools that we've worked on this month, and start building tests using those tools. We also hope to begin work on a firm 1.0 roadmap for Toga.

Want to get involved?

Want to get involved? Here's 8 open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:

  1. Modify the handling of app signing options
  2. Filter out a message generated after Xcode updates
  3. Add the ability to refresh the support package in a project
  4. Provide protection against creating projects with "semi-reserved" names
  5. Add the ability to configure the ABIs built by an Android project
  6. Add support for the ANDROID_HOME environment variable
  7. Add a command summary to briefcase --help
  8. Add an option to display the platforms supported by a command

Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.

October 2022 Status Update

Posted by Russell Keith-Magee on 1 November 2022

How time flies! Progress in October was a little slower than in past months; preparing for and attending DjangoCon US, plus some well-earned holiday leave reduced the amount of time we had to work on new features. However, we were able to make a number of significant improvements.

What we've done

During October:

  • We restored the web backend for Toga! Web deployment has always been part of the BeeWare story, but we hit some technological challenges with the approach that we were taking in the Toga 0.2 branch. However, with the addition of WASM as an officially supported platform in CPython 3.11, and the release of PyScript, we have a new way of getting Python in the browser. We have been able to use these tools to build a much improved Toga web backend.
  • We added support for deploying Briefcase apps as static web sites.
  • We released official support for Python 3.11 on macOS, iOS and Linux - on the day that Python 3.11 was released!. Python 3.11 support for Android and Windows is ready to go, and should be published in a day or two.
  • We modified Briefcase to write its logs into subdirectory, instead of littering them in the project directory.
  • We presented a talk at DjangoCon US 2022. The video for this talk should be up in around a month. A major feature of this talk was a demo of an Electron-style "web site as an app", built entirely in Python. This demo (called Positron) has been included in the Toga repository as an example.
  • We attended the DjangoCon US 2022 sprints, and handed out 4 challenge coins to new contributors!
  • We added the ability to retrieve widgets by ID to the Toga API.
  • We made a big change to the way that Toga backends are discovered. This new approach has two major advantages - firstly, it means third-parties can register their own backends; but more importantly, it means that code no longer needs to explicitly provide a backend. This simplifies the creation of test cases - but it also means that icons, images and fonts no longer need to be "late bound". This is an internal detail, but it significantly simplifies the code using those data types.
  • We added the ability to create Image objects from raw data, rather than an image file.
  • We started the process of modernising the layout and tooling of the Toga repository. Some of these changes have already landed; more will come in the coming days. This ensures that we have a stable foundation for future work on Toga.

What's next?

During November, we continuing to focus on Toga, with a particular focus on testing. A particular focus will be working out how to test graphical features of Toga - an area that has historically been managed entirely with manual testing. Having strong automated testing of graphical features will be a key milestone in ensuring Toga remains stable in the long term.

Want to get involved?

Want to get involved? Here's 8 open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:

  1. Improve protection against corrupted or incomplete downloads
  2. Modify the handling of app signing options
  3. Filter out a message generated after Xcode updates
  4. Add the ability to refresh the support package in a project
  5. Provide protection against creating projects with "semi-reserved" names
  6. Add the ability to configure the ABIs built by an Android project
  7. Add support for the ANDROID_HOME environment variable
  8. Silence an warning that is displayed when an Android emulator has no skin defined

Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.

2022Q4 Roadmap

Posted by Russell Keith-Magee on 4 October 2022

With Q3 completed, it's time to provide an update on the long term goals and priorities of the BeeWare project. As always, this roadmap should be read as a guide to what we aim to focus on over the coming quarter, rather than a hard commitment of features that will be made available on a specific deadline.

Q3 progress

We were able to hit our biggest Q3 goals:

  • Linux packaging - We've investigated and addressed all the known issues with Linux packaging. There's one pending enhancement that would expand the range of third-party binary packages that can be included in Briefcase projects.
  • Windows packaging - We've introduced a stub binary for Windows apps (as well as a Visual Studio target for Windows projects). This allowed us to significantly improve log capture, so when a Windows app crashes it's a lot easier to obtain diagnostic information. This change also corrected most of the issues with the way Windows apps were using the site module; however, there is still one issue related to the handling of packages that have complex post-installation processes.
  • Binary modules - By far the biggest achievement of Q3. With the release of Briefcase 0.3.10, we now support binary modules on iOS and Android. We have also published 40 of the most popular binary modules, compiled for iOS and Android.

However, we didn't hit all our goals:

  • Testing - Testing support has improved substantially. We have a much more thorough CI configuration on Briefcase that verifies that templates generate valid projects. We also have a testbed project that we can use to check that support packages are behaving correctly. However, we are not yet able to run this test project in CI, so we don't have automated validation of PRs against the support packages. We also made very little progress in GUI testing.
  • Conda integration - We began initial discussions around Conda integration, on two fronts: firstly, to investigate the possibility of using Conda as a base for package management in apps; and secondly, about using conda-forge as a framework for managing community contributions. Both of these conversations are ongoing; it will likely take some time before these efforts bear fruit.

Q4 priorities

In Q4, we're planning to focus on:

Toga

The primary focus for Q4 will be Toga. There are a number of pieces of work involved with this:

  • Housekeeping - Basic repository maintenance tasks, like enabling black, introducing pre-commit hooks, towncrier, and automating release infrastructure.
  • Example consolidation - Toga's examples folder has become a sprawling collection of code that is part documentation, part testbed. We need to start consolidating these examples into 2 apps - one that is a user-facing demonstrator of Toga features, and one that can be used as a functional test of Toga features.
  • Testing - Modernising and cleaning up the existing test suite, and working out how to automate GUI tests
  • Documentation - Toga's documentation has lagged behind an acceptable standard for a long time,
  • Planning - Developing a clear plan for what will be in Toga 1.0
  • Cross-platform hardware - While it has always been technically possible to support the APIs needed to access device hardware, there hasn't been a cross-platform API for this work. In Q4, we hope to demonstrate at least 1 interesting piece of mobile device hardware.

Although we won't get to Toga v1.0 in Q4, we will land a v0.3 final, after several years of maintaining dev pre-releases as the main way to use Toga.

Web support

Toga has historically had support for the Web as a platform; however, that effort was put on hold because the approach we were taking wasn't proving viable. However, with the release of Python 3.11, WASM becomes an officially supported CPython platform. We've also seen the release of PyScript, demonstrating that client-side Python in the browser is an extremely viable proposition. The time has come to resurrect BeeWare's support for the web as a deployment platform. This will involve getting Toga's web backend into a working state, and adding a web deployment backend for Briefcase.

Longer term goals

Looking into next year, the big items on the horizon are:

Upstreaming

With binary modules now being supported on both iOS and Android, and the introduction of a Tier 3 support level in PEP11, the prospect of elevating iOS and Android to official supported platforms is seeming increasingly plausible. Our hope is to go to PyCon US next year in a position to make a concrete proposal to the CPython core team.

Toga 1.0

The broad goal for "Toga 1.0" could be considered "Tkinter, but with some additional bells to show off". We want to deliver a core widget set of the "essentials", plus some nice features to show off, like WebViews and camera or GPS integration. We hope to establish the feature list for Toga 1.0 in Q4; once that plan is in place, we'll have a better idea of when we will be able to deliver on that plan.

A Forge for mobile packages

As of today, all binary modules for mobile platforms are being maintained by the BeeWare team. This isn't a viable solution long term; we need to move to an environment where the community can manage binary packages for mobile platforms.

Summary

For the last 6 months, the focus of BeeWare has been on Briefcase and the support packages - ensuring that we have a feature complete, robust set of tools for deploying apps on any platform. In Q4, our focus is shifting to the tools we need to build apps on those platforms, and ensuring we have a stable foundation for future development. It's taken a long time for BeeWare to get this far - but we're getting very close to unlocking the remarkable potential of the Python ecosystem on mobile devices.

September 2022 Status Update

Posted by Russell Keith-Magee on 3 October 2022

September has been a big month for the BeeWare team. There have been lots of small updates - but there's one very large, eagerly anticipated update that has taken several months to complete.

What we've done

During September:

  • We added support for binary packages on mobile platforms! There's more detail related to this than can fit in a bullet point; more details below.
  • We added the ability to remove files from a templated app. This can be used to trim down a project, removing parts of the Python interpreter that you don't need (such as standard library modules that you don't need)
  • We've changed the way we use templates, switching to using branches based on the Briefcase release, rather than the Python release. This enables us to make development changes to templates without affecting the published stable release.
  • We reworked the way Briefcase manages integrations with the tools needed to build apps. This was a mammoth project, but as a result the process of verifying tools is faster, and a number of bugs related to having stale builds have been resolved.
  • We've made a number of improvements to console handling of log messages, especially on Windows
  • We improved error handling in Toga when TLS1.2 or TLS1.3 isn't supported by Windows.
  • We started conversations with the teams at Anaconda responsible for conda about using Conda as a base for Briefcase apps, and on using Conde-forge style infrastructure to support binary packaging. Nothing concrete has come from these conversations yet; these conversations are ongoing.

Binary package support you say?

All the new binary packaging support is contained in Briefcase 0.3.10.

On Android, binary module support was achieved by moving to Chaquopy as a base for Android projects. Along with support for binary modules, this change has the added benefit of faster startup times, and less bugs at runtime. This means you'll need to re-generate any Android project - the Python source code should work as-is, but there's no simple way to convert the Android Gradle project from the older Rubicon base to the newer Chaquopy base.

On iOS, we've reworked how the support packages are constructed, and added support for installing binary modules from a Beeware-specific package repository that includes builds of iOS binary wheels. We've also added asset signing to the iOS Xcode project - and applied the same changes to the macOS backends. Of course, this also means you'll need to re-generate any iOS project to take advantage of these changes.

While we are now able to support binary packages on mobile platforms, this doesn't mean that every binary package is automatically supported. Binary wheels need to be compiled for mobile platforms, and compiling for mobile platforms can be complicated. However, we've compiled ~40 of them most popular packages with binary components (including numpy, pandas, cryptography, pillow and matplotlib), and made them available as dependencies. In order to get working builds, some of these packages are slightly older versions (e.g., cryptography is a version from before the introduction of the rust dependency); however, we've endeavoured to provide the most recent versions possible.

A full list of supported packages can be found on the Chaquopy package repository for Android, and the BeeWare repository for iOS. If your project uses one of these packages, you only need to add them to the requires definition in your pyproject.toml, then update and re-build your project, and you're done.

If there’s a package you want that isn't on this list, you'll need to request a binary wheel. For Android, open a ticket on the Chaquopy repository. For iOS, open a ticket on this repository. If you're adventurous, you can also try building wheels yourself. The issue trackers are on a project that has as "forge-like" tool for building wheels, in the server/pypi folder; there are tools and documentation describing how to build a package. The use of these projects is a short-term measure; in the longer term, we want to move to an actual “forge”-style community-managed model for managing binary packages.

What's next?

For the last few months, the focus of the BeeWare team has been on Briefcase and the support packages. In October (and the rest of Q4), we'll be shifting our focus to Toga. The initial focus will be on improving the foundation of the project - testing and build infrastructure; but once those pieces are in place, we'll start adding new features.

We'll also be at DjangoCon US in San Diego - and there will be some web-related updates related to the presentation that will be given at that conference. We hope we'll see you there!

Want to get involved?

Want to get involved? Here's 10 open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:

  1. Write Briefcase logs into a subdirectory, instead of the project folder
  2. Improve protection against corrupted or incomplete downloads
  3. Modify the handling of app signing options
  4. Modify the Briefcase new project wizard to show which repository and branch it's using
  5. Filter out a message generated after Xcode updates
  6. Add the ability to refresh the support package in a project
  7. Provide protection against creating projects with "semi-reserved" names
  8. Add the ability to configure the ABIs built by an Android project
  9. Add support for the ANDROID_HOME environment variable
  10. Silence an warning that is displayed when an Android emulator has no skin defined

Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.