July 2023 Status Update
The year rolls on, and so does progress on BeeWare!
What we've done
- We attended EuroPython 2023, presenting on Briefcase, attending the WASM summit, and running a very successful 2 day sprint.
- We've completed the audit of all Toga widgets on macOS, iOS and GTK! We've also completed the audit of ScrollContainer and SplitContainer on Windows and Android.
- We've added support for Docker Desktop and rootless Docker on Linux.
- We've added the ability to get detailed logs from the tools used by Briefcase to build apps.
- We've corrected some warnings generated by pip when installing Python packages on Linux.
- We've added support for generating APKs as an output format for Android.
- We've improved error reporting when an application can't start on Android due to the device being too old, or the Android device clock isn't in sync with the developer's laptop.
- We added __repr__ and __str__ implementations for Objective C structures like NSRect, NSRange, NSPoint and more.
- We formally dropped support for Python 3.7 in Toga.
- We improved error handling when an app doesn't define a main window as part of it's startup.
- Type annotations in Toga have been significantly improved. We've been adding type annotations as part of the widget audit, but some types (such as callbacks) weren't as specific as they could have been. We're now using Protocols to define some of the more complex types in Toga.
- We added an ActivityIndicator and Divider widget implementation in the Web backend.
What's next?
The widget testing audit is now complete on macOS, iOS and GTK. An audit of App and Window functionality is all that stands in the way of 100% test coverage on those three platforms; it seems likely we'll get there by the end of this month. Android and Windows coverage is close behind, but might take a little longer.
Part of the reason for this delay is that we need to address an important change in the most recent release of Android Studio. Over the last few years, the Android ecosystem has been in the process of migrating its build system from Groovy to Kotlin; Android Studio Giraffe makes Kotlin the default for new projects, so we need to make sure we're compatible with that change. The widget audit has also highlighted that we need to improve our handling of subclass inheritance in Java; we're hoping to make some changes that will enable us to fill in a few more gaps in widget API coverage on Android.
We'll also be at PyCon AU 2023 from August 18-22. We're presenting on Saturday; and we'll be there for the full duration of the sprints. See you in Tarntanya/Adelaide!
Want to get involved?
Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:
- Filter out a message generated after Xcode updates
- Add the ability to configure the ABIs built by an Android project
- Rationalise the application of adhoc signing on macOS
- Provide an option to override app settings from the command line
- Add support for custom PyPI repositories
- Document how to debug an application in popular IDEs
- Switch Toga's Winforms backend to use native Winforms imports
- Switch Toga's Android backend to use native Chaquopy syntax
Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.
2023Q3 Roadmap
2023 is halfway done, and BeeWare continues to make steady progress. As always, this roadmap should be read as a guide to what we aim to focus on over the coming quarter, rather than a hard commitment of features that will be made available on a specific deadline.
Q2 progress
As with Q1, our focus in Q2 was on Toga testing. This has been a mammoth project; but it's finally nearing completion. All but 2 widgets (Canvas and DetailedList) have 100% branch coverage on macOS, iOS and GTK, while Android and Windows are only 5 more widgets behind (ScrollContainer, SplitContainer, OptionContainer, Table and Tree). Along the way, we've identified and resolved lots of inconsistencies in widget APIs, and resolved countless bugs. We've also fully documented every widget, ported the test suite to Pytest, and added type annotations in the code (mostly for documentation purposes, rather than formal validation). We've also completed testing and coverage of a lot of the non-widget functionality that wasn’t strictly planned for this quarter - features like Images, Icons, Validators, application paths, and colors; plus a lot of tests (and bugfixes) in the core widget layout. We've also got the start of some tooling that can be used to test the Web backend.
Unfortunately, we haven't been able to do any substantial Toga planning. Once Toga's code is at 100% testing coverage, we'll be in a much better place to develop those plans. We also haven't been able to add any native cross-platform hardware support.
Q3 priorities
Our Q3 goals are:
- Finalize Toga's test suite. By the end of Q3, we should have 100% branch coverage on every line of Toga's core, and on the macOS, Windows, GTK, Android and iOS backends.
- iOS Binary packaging We've been made aware that Apple is rejecting Briefcase packaged iOS apps from the iOS App Store because of a package layout issue associated with binary modules. This is a very high priority to fix.
- Kotlin upgrades The upcoming Android Studio release is switching to Kotlin as its preferred language for new Gradle configuration files; we need to upgrade Chaquopy's tooling to support this.
- Add full support for Java inheritance In the process of building Toga's test suite, it's become clear that a number of features can only be implemented by subclassing Java base classes. Chaquopy's ability to subclass is currently limited; to unlock all the features of Android, we need to resolve this limitation.
Longer term goals
Our longer term goals are also largely unchanged. Toga 1.0 remains an important goal; once we're at 100% test coverage, we will be in much better place to provide estimates about Toga 1.0. Upstreaming is an area where we have some good news - as a result of discussions at Pycon US, we've received support for raising iOS and Android to the status of Tier 3 supported platforms in CPython, with the initial goal of having this in place for Python 3.13. You should expect to see movement on upstreaming towards the end of this year, and early next year. A forge for mobile packages is also remains an important goal; the completion of upstreaming work will make the goal of building a forge much more viable.
Summary
At the end of Q2 2023, Toga's testing journey is nearing completion... but there's still lots of work to be done. However, the groundwork that we've been doing over the last 6 months has put us in a very strong position to develop Toga and the rest of the BeeWare ecosystem going forward.
June 2023 Status Update
We're half way through the year! Work on the widget audit is almost complete; but that's not all we've been working on.
What we've done
- We've completed the audit of NumberInput, WebView, DateInput, TimeInput, ImageView and Selection on all platforms; and we've completed the audit of ScrollContainer, SplitContainer, OptionContainer, Table and Tree on macOS, iOS and GTK.
- We've identified and resolved some edge cases in the Pack layout algorithm.
- We've improved the implementation of application paths, and audited the paths being returned on all platforms.
- We corrected some issues in the Web backend caused by the most recent PyScript and Pyodide releases.
- We've improved the error message that is returned when a widget isn't implemented on a platform.
- We've updated Briefcase to use Java 17 when building Android projects.
- We've added support for the ANDROID_HOME environment variable when building Android projects with a pre-existing copy of Android Studio.
What's next?
The widget testing audit has been a mammoth project, but the end is very nearly in sight. There's only 2 widgets (Canvas and DetailedList) that haven't seen any testing to date; plus there's some non-widget features (like dialogs, fonts, windows, and basic App functionality) that still require testing. Once those tests are complete, we'll be moving on to dealing with some iOS and Android deployment issues that have been accumulating while we focused on Toga.
We'll also be at EuroPython 2023 from July 17-23. We're presenting on Thursday; and we'll be there for the full duration of the sprints. See you in Prague!
Want to get involved?
Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:
- Filter out a message generated after Xcode updates
- Add the ability to configure the ABIs built by an Android project
- Rationalise the application of adhoc signing on macOS
- Detect a failure when trying to install on an Android device that is too old
- Use the on-device clock to filter Android device logs
- Add an APK packaging target for Android
- Provide an option to override app settings from the command line
Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.
May 2023 Status Update
May has been another month of steady progress - mostly on widget testing, but also in some other key areas of the project.
What we've done
- We've completed the audit of the TextInput, PasswordInput, and MultilineTextInput widgets; we've completed the audit of NumberInput, WebView, DateInput, TimeInput, and Selection on macOS, iOS and GTK; and we've begun the audit of ImageView.
- We've added documentation and tests for the input validation API as part of the TextInput audit.
- We've added documentation and tests for ListSource as part of the Selection audit, and modified the implementation of Selection to use a ListSource.
- We refactored the implementation of Dialog so that it is architecturally consistent with the rest of Toga; this will hopefully also lead to an easier testing story.
- We made significant progress in supporting custom fonts on GTK, including adding a GUI test of this capability.
- We completed a refactor of Briefcase's tool integrations. This introduces a consistent API for all tools that Briefcase uses, as well as introducing type annotations for a lot more of the codebase.
- We've introduced conditional coverage to the Briefcase codebase. This means developers who only have access to a single testing platform can get a 100% coverage report on their work before submitting to CI for cross-platform confirmation.
- We've helped triage a number of bugs in the Python 3.12.0b1 release (revealed by our testing for Briefcase and Rubicon-ObjC).
- We've modified Briefcase's Flatpak backend to require an explicit runtime, rather than falling back to a default runtime that is no longer supported.
- We developed a draft implementation of a Briefcase test runner for the Web backend. This is an important first step to being able to unit test Toga's Web backend; however, there's still a couple of technical challenges outstanding before we can complete that work.
- We've begun the process of upgrading Briefcase's Java support from Java 8 to Java 17.
What's next?
June will be another month with a focus on GUI test coverage. We're still hoping that this work will be complete by the end of June - we're averaging 1-2 widgets a week, and we have 8 widgets remaining; however, every widget is throwing up unique challenges (and revealing deep bugs - sometimes in the underlying GUI frameworks), so it's possible the audit work may need to continue into July.
Want to get involved?
Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:
- Filter out a message generated after Xcode updates
- Add the ability to configure the ABIs built by an Android project
- Add support for the ANDROID_HOME environment variable
- Rationalise the application of adhoc signing on macOS
- Detect a failure when trying to install on an Android device that is too old
- Use the on-device clock to filter Android device logs
- Add an APK packaging target for Android
- Provide an option to override app settings from the command line
Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.
April 2023 Status Update
April was dominated by preparation and attendance at PyCon US 2023 - but we've seen some good progress over the month.
What we've done
PyCon US saw a lot of BeeWare activity:
- We presented 2 talks at PyCon US - one on Briefcase, and one on the state of Android support on Python. Videos of these talks should appear on the PyCon US Youtube channel in the near future.
- We presented at the Language Summit, where we received provisional approval to push for iOS and Android to become Tier 3 supported platforms in CPython 3.13.
- We presented at the Packaging Summit, providing the needs of packaging from a mobile perspective.
- We had a booth in the community section of the conference show floor.
- We ran a successful sprint where we handed out over a dozen Challenge Coins. Many of the improvements listed in this status update come from community contributions, and we have a backlog of other PRs we're still working on.
- We donated an leather Briefcase to the PyLadies Auction, raising US$800!.
In non-PyCon activity:
- We added the ability to code sign Windows release artefacts.
- We added a new distribution format for Windows apps - simple ZIP archives.
- We modified Briefcase to support PEP621 metadata standards. Briefcase predates the definition of PEP621, but there's a lot of overlap in what PEP621 defines and what Briefcase defines, so it makes sense for Briefcase to adhere to the common standard. In future, we expect to see more convergence with PEP621 standards.
- We landed major updates to Briefcase's AppImage and Flatpak backends. They now use Indygreg's Standalone Python; this significantly speeds up Flatpak builds, should be more stable on AppImage, allows us to switch to manylinux base images - and it reduces the number of projects that BeeWare is responsible for maintaining.
- We got 2 more widgets to 100% GUI test coverage. One of these is the base widget, so we're now able to guarantee basic functionality across all widgets.
- We migrated Toga's web backend to use the Shoelace web components framework. This is a much better match for Toga than Bootstrap was, as it is primarily focussed on rendering widgets, rather than HTML layout. We only have a handful of widgets ported over at this point - but the process of implementing widgets is relatively straightforward, so if you have web skills and are looking for a way to contribute, this would be a great place to get involved.
- We added support for cursor visibility, full screen mode and determining the current window on Windows.
- We added an API for catching a 'confirm' action on TextInput widgets.
- We added support for FreeBSD in Toga.
- We made a number of significant improvements to the Tox configurations to improve reliability, work around a bugs in the release of pip 23.1, speed up and parallelise test execution, enforce spelling and external linking in documentation, and to make easier to obtain coverage reports.
What's next?
In May, we'll be continuing our push for GUI test coverage. Our current aim is to have complete widget coverage by the end of June.
Want to get involved?
Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:
- Modify the handling of app signing options
- Filter out a message generated after Xcode updates
- Add the ability to configure the ABIs built by an Android project
- Add support for the ANDROID_HOME environment variable
- Rationalise the application of adhoc signing on macOS
- Detect a failure when trying to install on an Android device that is too old
- Use the on-device clock to filter Android device logs
- Add an APK packaging target for Android
- Provide an option to override app settings from the command line
Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.
2023Q2 Roadmap
It doesn't feel like it's been three months since our last roadmap update... but here we are, in April 2023. As always, this roadmap should be read as a guide to what we aim to focus on over the coming quarter, rather than a hard commitment of features that will be made available on a specific deadline.
Q1 progress
Our focus this quarter has been on testing; however as with our last report, our progress has not been as significant as we hoped it would be. Getting the first tests in place has proved to be much more complex than we anticipated. In order to test widgets at all, we've had to set up all the infrastructure that will be necessary to test every other widget, such as defining the probes to inspect colors and fonts, and to ensure that the GUI framework has finished applying changes before performing a test assertion. The process of building the first tests has exposed a number of problems deeply embedded in the core of Toga - problems that either aren't observable or are easily masked when you're writing an application as a user, but can't be avoided when you're writing a test suite that will run fast and needs to have repeatable behavior. We're also testing the limits of the features exposed by each platform - and everything we do needs to be repeated on 5 GUI toolkits with 5 completely different views of the world.
The good news is that as of the end of Q1, we're finally getting traction on the problem. We're about a quarter of the way through the widgets by count. Admittedly, we've been tackling the simpler widgets first - but the lessons learned (and bugs fixed) making those simple widgets work should make testing the more complex widgets easier. There's also some commonality in the more complex widgets - for example, TextInput, PasswordInput and NumberInput are all essentially the same widget, so once tests are built for one of them, the other 2 should fall out reasonably easily.
We have also been able to release Toga v0.3.0. This release was almost 5 years in the making.
Q1 saw some unexpected improvements to Briefcase. We've had persistent problems supporting AppImage and Flatpak as packaging formats; during Q1, we couldn't avoid these problems any longer, so we introduced a System packaging backend, and made this the new default for Linux apps. We're now able to produce native system packages for Briefcase apps, supporting almost every Debian, Red Hat or Arch derived distribution. These packages are already proving more stable: we had no major issues during a recent tutorial at Everything Open 2023, despite a wide variety of Linux distributions in the room - something that we've never been able to say about AppImage.
However, once again, that means there's a lot left on the Q1 TODO list. Toga documentation is being slowly updated as we audit each widget; however, we haven't made any changes to the example apps; we haven't been able to do any substantial Toga planning; and we haven't been able to add any native cross-platform hardware support.
Q2 priorities
Our short term goals are, therefore, largely unchanged from last quarter:
- Testing - Completing the Toga GUI testing suite will again be our primary focus for the quarter.
- Planning - Once we've got the test harness in place, we'll be in a much better position to evaluate what work remains to get to v1.0.
- Documentation - The process of systematically building a test suite also gives us an opportunity to update the API documentation for each widget as we go. There's still lots of documentation required on top of the API guide, but complete API documentation will be a big start.
- Cross-platform hardware - Cross-platform support for device hardware (especially mobile) is one of the banner features that makes mobile device support worthwhile. Providing proof that this hardware support is easy to use will be a key demonstrator for BeeWare as a platform.
Longer term goals
Our longer term goals are also largely unchanged. Upstreaming, Toga 1.0, and a forge for mobile packages are all important goals for the BeeWare ecosystem. However, we don't expect to make any significant progress on most of these goals until we have solidified Toga's testing story. We should, however, see at least the first signs of progress on upstreaming BeeWare's work on mobile as a result of PyCon US, where we'll have a chance to raise issues directly with the Python core team.
Summary
Providing a robust GUI testing framework for Toga has definitely proven to be a lot more complicated than we originally thought. The good news is that as of the end of Q1 2023, we're finally starting to make good progress on testing. We hope that by the time of our next status update, we're able to report that we're nearing the end of our testing journey, and we can turn our focus to other areas of interest.
March 2023 Status Update
March has been a month of slow but steady progress for BeeWare.
What we've done
- Progress on GUI testing in Toga is continuing. We now have 7 widgets fully tested, with another 2 in progress. As with last month's update, getting these tests passing has required wrestling a lot of fundamentals, in particular around the simulation of events such as mouse clicks. We're also discovering (and fixing) lots of corner cases in widget implementations that only surface when you're doing rigorous and repeatable testing.
- We've made some major changes to the way GTK widgets are rendered. This change significantly reduces the amount of re-rendering that is performed by GTK, as well as resolving a number of long standing issues related to resizing widgets.
- Thanks to some community contributions, we were were able to add support for focus events and text alignment on GTK's TextInput widget, and support for text alignment and fonts on GTK's NumberInput widget.
- We landed the implementation of a Linux system package backend for Briefcase, which can output DEB and RPM packages. Thanks to a community contribution, we were also able to add support for Arch packages
- We modified the locations where Briefcase outputs build artefacts. This removes some sources of bugs associated with tools not supporting spaces in filenames, and makes Briefcase's output consistent with other tools in the Python ecosystem.
- We made significant progress on modifying Briefcase's AppImage and Flatpak backends to use Indygreg's Standalone Python, rather than BeeWare's Linux support package. Standalone Python is becoming a de facto standard for pre-compiled Python builds (especially on Linux), so it makes sense for BeeWare to converge on this common standard. This change will also significantly speed up Flatpak builds, should be more stable on AppImage, allows us to switch to manylinux base images - and it reduces the number of projects that BeeWare is responsible for maintaining. We're currently waiting on a fix to the linuxdeploy GTK plugin to be merged before we can land this change.
- Thanks to a community contribution, we added support for PyGame to Briefcase.
- We migrated BeeWare's CI to use the new System packaging backend on Linux. This means that CI is significantly faster, as building system packages requires a lot less processing than an AppImage.
- We released Travertino 0.2.0. Travertino is the library that underpins Toga's Pack layout algorithm. It's been several years since we last did a Travertino release, but some recent bug fixes warranted an update.
- We presented a tutorial at Everything Open 2023.
We also made a big change to our social media strategy: we started a Mastodon account. We've wound down our Twitter usage at the end of last year; going forward, you can get your bite size BeeWare updates from @beeware@fosstodon.org.
What's next?
In terms of technical goals, April will continue our push for GUI test coverage. However, we're expecting progress to be a little slower, because of one big event - PyCon US. We'll have a booth in the community section of the conference floor; the core team are presenting 2 talks; we're presenting at the Language Summit; and we'll be at the sprints for the first 3 days. Preparing for these talks will take up a lot of our time in April, as will the event itself - so progress is likely to be a little slower than normal. If you're going to be there, come by the booth and say hi! If you'd like to help us to staff the booth - get in touch on Discord, on Mastodon, or email!
Want to get involved?
Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:
- Modify the handling of app signing options
- Filter out a message generated after Xcode updates
- Add the ability to refresh the support package in a project
- Provide protection against creating projects with "semi-reserved" names
- Add the ability to configure the ABIs built by an Android project
- Add support for the ANDROID_HOME environment variable
- Improve validation of project names in the Briefcase wizard
- Rationalise the application of adhoc signing on macOS
- Correct the capture of log messages on the web backend
Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.
February 2023 Status Update
Although there haven't been a lot of new BeeWare features developed in February, the work that has been completed represents some important progress against our overall project goals.
What we've done
- The biggest milestone is that we have achieved 100% test coverage for our first widget - Button! This may sound a little underwhelming - after all, Toga has had a Button widget for a long time, and has been fairly reliable. However, getting to this point has required a lot of smaller (seemingly unrelated) problems to be solved. It has also required setting up all the infrastructure that will be necessary to test every other widget, such as defining the probes to inspect colors and fonts, and to ensure that the GUI framework has finished applying changes before performing a test assertion). We've also taken this opportunity to start migrating the Toga unit tests to pytest, which promises to significantly improve our testing capabilities.
- We formalised the relationship between Toga's Pack layout model and CSS. This should make it a lot easier to determine what "correct" layout behavior should be.
- We identified a low level issue with the way that layout is performed on GTK, and possibly with other platforms. We're still working through the consequences of this change, but it has indirectly resolved a number of issues we have seen reported with widget layouts, especially on GTK.
- We've added the ability to pass command line arguments to running apps from the briefcase dev and briefcase run interface. This was primarily done to make it easier to run unit test suites; but it can be used for any other reason you might want to use a command line argument.
- We have a draft implementation of a Linux system packaging backend for Briefcase. We've had consistent problems with both AppImage and Flatpak, most of which can be traced back to resolving links to system packages. We're not going to remove these backends - when AppImage and Flatpak work, they're great options - but we're going to change the default packaging backend on Linux to be system packages (i.e., DEB/RPM etc). This makes the simple case of building a package for your own system much easier, while retaining the ability to build packages for other distributions as required. We're hoping to land this PR in the coming month.
- We corrected an issue with threading modes that prevented some Windows apps (especially PySide/Qt apps) from displaying dialogs.
- We corrected an issue with the Briefcase Web backend that would cause intermittent errors when loading a web app.
- We updated our CI infrastructure to make better use of shared task definitions. BeeWare has a lot of Github repositories, and a lot of common tasks (such as rolling out an app across multiple Python and operating system versions). Previously, each Github repository had an independent set of CI task definitions; we've now got a central collection of task definitions that is used by all repositories.
- We cleaned up the table in Toga's documentation describing the state of support of each widget. This allows us to differentiate between widgets that we've fully tested, and widgets that exist, but haven't been tested.
- Thanks to a community contribution, we've been able to publish a backlog of iOS binary packages for Python 3.8-3.11. For the packages we currently support (including Numpy, Pandas, Pillow and more), we're now able to offer an iOS binary wheel for every published version for which there is a corresponding macOS wheel, going back to 2019.
What's next?
In March, we'll be continuing to improve GUI test coverage. Now that we've got one widget to 100% coverage, we know we have all the core testing infrastructure in place, so it should be much easier to implement tests for the future widgets. By the end of March, we should have a lot more widgets at 100% coverage.
We'll also be presenting at Everything Open 2023. If you're attending, come say hi!
Want to get involved?
Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:
- Modify the handling of app signing options
- Filter out a message generated after Xcode updates
- Add the ability to refresh the support package in a project
- Provide protection against creating projects with "semi-reserved" names
- Add the ability to configure the ABIs built by an Android project
- Add support for the ANDROID_HOME environment variable
- Add a template branch option when creating a new project
- Improve validation of project names in the Briefcase wizard
- Rationalise the application of adhoc signing on macOS
- Correct the capture of log messages on the web backend
Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.
January 2023 Status Update
BeeWare has had an exciting start to the new year, with a couple of significant releases
What we've done
We had two big releases in January.
Firstly, we released Briefcase 0.3.12. This is the first Briefcase release in 3 months, so it includes a lot of improvements, but the most notable improvements are the new "Testing mode", a major improvement to the Android build process, and a huge number of bug fixes and improvements - especially on Windows.
Secondly - and more importantly - we released Toga 0.3.0! No more dev releases or --pre flags for pip - the stable release of Toga on PyPI is now closely tracking actual development. It's been over 5 years since we last released an official stable Toga version, even though we've been recommending the dev releases for most of that time. However, as a result of a number of changes that landed this month, we're finally committing to the 0.3.0 branch properly.
These releases incorporate changes going back months (and years); but we made some very specific progress during January:
- Chaquopy had a new release., significantly improving build times on Android, improving the rendering of stdout content in the Android logs, adding some features that make it easier to expose the original project source code in the running app bundle, and enabling PEP517 builds of project dependencies.
- Briefcase is now able to reference local source packages when building Linux AppImage binaries.
- Some errors caused by the way the Windows Store version of Python handles local application data have been corrected.
- Briefcase's help text now shows the list of platforms and output formats, rather than requiring an additional help call to retrieve that information.
- Some issues with thread and asyncio safety in Rubicon ObjC were identified and resolved, along with an issue associated with the way some Objective C classes (such as NSWindow) are renamed at runtime.
- Initial GUI testing backends for macOS, iOS and GTK were added.
- A long standing issue with the replacement of root window content on GTK and iOS has been resolved.
- Coverage testing was added to the Toga Android GUI test backend.
- An API was added to allow scrolling MultilineTextInput widgets to the top and bottom of the widget content.
- The work moving Toga to use Setuptools-SCM was completed. Development versions of Toga will now report a specific version control commit; release versions have their version determined by the git tag used to produce the release.
- We completed a major review of the way we're using Github issue labels, and modified those labels to be consistent across all repositories.
What's next?
Much of the last 3 months have been spent getting a GUI testing framework in place for Toga. That work is now in place, so now we'll be working full time on writing GUI tests for Toga, with the aim of getting Toga to 100% coverage. Along the way, we'll discover all the edge cases, bugs, and partially missing implementations that are lurking in the individual Toga widgets.
Want to get involved?
Want to get involved? Here are some open issues that would be a great place to get started with contributing to a BeeWare project. They're all relatively minor changes, but would provide a big improvement to the lives of BeeWare users:
- Modify the handling of app signing options
- Filter out a message generated after Xcode updates
- Add the ability to refresh the support package in a project
- Provide protection against creating projects with "semi-reserved" names
- Add the ability to configure the ABIs built by an Android project
- Add support for the ANDROID_HOME environment variable
- Purge cache folders when installing app code in Briefcase
- Add a template branch option when creating a new project
- Improve validation of project names in the Briefcase wizard
Pick one of these tickets, drop a comment on the ticket to let others know you're looking at it, and try your hand at a PR! We have a guide on setting up a Briefcase development environment; but if you need any additional assistance or guidance, you can ask on the ticket, or join us on the BeeWare Discord server.
2023Q1 Roadmap
Welcome to a new year! 2022 saw some amazing progress for BeeWare; and we've got some big plans for 2023. As always, this roadmap should be read as a guide to what we aim to focus on over the coming quarter, rather than a hard commitment of features that will be made available on a specific deadline.
Q4 progress
Progress in Q4 was not as significant as we originally planned. A combination of conference travel and holiday leave meant the quarter was shorter than most; but even then, our goals for the quarter were extremely optimistic.
We were able to hit our goal for Web deployment. We've restored the Toga web backend to a working state, and we've added a Web deployment backend to Briefcase. There's still a lot of work required before Web deployment is a viable as a production solution, but the initial pieces are now in place.
We were also able to hit our goals on housekeeping. We've made a lot of improvements to the base infrastructure of repositories, enabling Black, pre-commit, and towncrier in Toga, and improving coverage reporting (including getting to 100% branch coverage in Briefcase's test suite).
We've made significant progress on improving the testing story for BeeWare apps. We've added the ability for Briefcase projects to run normal Python test suites inside a packaged app, and we've demonstrated that we can use this capability to run CI for iOS and Android apps (as well as desktop platforms). This feature will be included in the next Briefcase release.
Using this new testing capability, we've started building a GUI test suite for Toga. This is a critical part of ensuring the quality of Toga going forward; but getting this testing framework in place has revealed lots of complications and bugs. We're making progress, but there's still a lot of work to be done.
Our plans to release Toga v0.3 final have also been delayed. The process of adding SCM-based versioning to the repository (part of the housekeeping task) revealed a bug that prevented code running on Android; fixing that bug became a blocker on the release of Toga v0.3. We've now resolved that problem, so we should be able to release Toga v0.3 final as soon as the fix has been released into the BeeWare's Android tooling.
Unfortunately, that leaves a lot on the Q4 TODO list. We haven't made any changes to the example apps; Toga documentation is mostly unchanged; we haven't been able to do any substantial Toga planning; and we haven't been able to add any native cross-platform hardware support.
Q1 priorities
It's probably unsurprising that Q1 2023 will be picking up on items we didn't get around to in Q4 2022.
- Testing - Completing the Toga GUI testing suite will be our primary focus for the quarter.
- Planning - Once we've got the test harness in place, we'll be in a much better position to evaluate what work remains to get to v1.0.
- Documentation - The process of systematically building a test suite also gives us an opportunity to update the API documentation for each widget as we go. There's still lots of documentation required on top of the API guide, but complete API documentation will be a big start.
- Cross-platform hardware - Cross-platform support for device hardware (especially mobile) is one of the banner features that makes mobile device support worthwhile. Providing proof that this hardware support is easy to use will be a key demonstrator for BeeWare as a platform.
Longer term goals
Our longer term goals are largely unchanged from Q4 2022. Upstreaming, Toga 1.0, and a forge for mobile packages are all important goals for the BeeWare ecosystem. However, we don't expect to make any significant progress on these goals until we have solidified Toga's testing story.
Summary
The seeming lack of progress made in Q4 was definitely disappointing; but the work hasn't been wasted. Progress has been slow specifically because the problems we're trying to solve are hard, and reveal every gnarled edge case that platforms have to offer. We are making progress, though; and once we've got the Toga testing framework stabilized, we'll be in a much stronger position to make predictions for future work on Toga.