Now Hiring!
BeeWare is now hiring!
The position is a full time, Mid- to Senior position. You will be working full time in the Open Source group at Anaconda, on the BeeWare suite of tools. Full details of the position can be found on Greenhouse.
The position calls for an unusual combination of skills. The ideal candidate would have experience building GUI applications (especially mobile) and Python skills. However, because of the existing state of the Python ecosystem, most Python developers don't have GUI development experience, and most GUI developers don't have extensive Python experience. For that reason, if the position is interesting to you, but you don't have all the "must have" attributes - I would encourage you to apply anyway. A candidate with no GUI development experience will still considered, as long as they've got a demonstrated history of doing weird and wonderful things with Python. Similarly, a developer with deep GUI experience, but no Python experience, will also be considered.
The job location requirements are also unusual. The position is remote; the position requires that your working hours need to be compatible with UTC+8. This means candidates from Australia, South East and South Asia will be a natural fit. European candidates will need to be prepared for early morning starts. US/Canadian candidates will need to be prepared for evening work (very late evenings if you're in CST or EST timezones). Anaconda has the capacity to hire in the UK, Germany, India, Australia, US, and Canada. If you're not a resident of one of those countries, it may be possible to hire you, but it will likely require you to operate as a private contractor rather than a salaried employee.
I'm incredibly excited for what the future holds for BeeWare - if you'd like to come on this journey with me, please apply (and tell them Russell sent you)!
April 2022 Status Update
One of the benefits of having the support of an organization like Anaconda is that we now have the time to perform little housekeeping tasks like publishing community updates. As a result, this is the first (of hopefully many) monthly updates letting you know what progress has been made in the BeeWare suite.
What we've done
During April:
- We've cleared the backlog of pull requests and issue triage. At this point, the only PRs that are open are ones that require updates or changes from the submitters. A lot of stale or duplicated issues have also been closed or consolidated.
- Webviews on Windows now use the EdgeChromium renderer, rather than the IE11 renderer. This massively improves the HTML compatibility of web views on Windows.
- Toga windows now use a "single source of truth" approach, using the API calls of the underlying GUI platform to determine the title, position and size of a window. This, in turn, allowed for the implementation of APIs that allow you to set the title, position and size of windows on desktop platforms.
- on_exit handlers on an App can now be asynchronous co-routines
- Toga dialogs that return a result (such as YesNoDialog) now use an asynchronous API. This is a backwards incompatible change that was required to support dialogs on Android; it also allows us to remove a nasty hack that was needed on iOS. However, we've been able to implement this in such a way that any existing usage that is incompatible with the new required usage will be flagged with a helpful error.
- The GTK web view has been improved, correcting some odd caching behaviour.
- Following the announcement of PyScript at PyCon US 2022, we've been able to resurrect the Web backend for Toga. This is still in the very early stages - but it's sufficiently advanced that we can run the first 2 tutorials, in both a "server side rendering" and "single page app" format.
- We've improved Briefcase's code signing integration on macOS. This should prevent some errors we've seen when signing apps that contain third-party binary modules
- We've significantly improved validation and error handling when an app name contains non-Latin characters , punctuation, or reserved words.
- We've added CI checks to ensure that Briefcase is always able to generate a valid signed app bundles
- We've improved the output generated by verbose debug output to improve full environment details . This should make it easier to identify why a command is failing due to unexpected user environments, and to assist in manually reproducing Briefcase's behaviour outside Briefcase.
- We now ensure that macOS apps are *always* signed. This is needed because all apps required at least ad-hoc signing to execute on M1 hardware.
Many of these PRs either began as, or are entirely the work of community members. There are also dozens of smaller PRs, bug fixes, and bug reports that are just as important, but are too numerous to list in a post like this. A huge thanks to everyone who has contributed to BeeWare during April.
What's next?
In May, we'll be focussing on:
- Updating the support packages for macOS and iOS apps. Apple's transition to M1 hardware means there are new simulator architectures that require support; this, in turn, requires that we adopt Xcode's new XCFramework for the packaging libraries, which requires some fairly major changes to the way the support packages are built.
- Updating the support packages on Android to support Python 3.10.
- Notarization on M1. This is becoming an increasingly required part of macOS app publishing, as Apple cranks up the security requirements.
We're also expecting to start the hiring process in the coming month. We'll provide more details when we can point a full job description. If you think you'd like to make BeeWare your day job (or you know someone who might), now would be a good time to start polishing your resume.
Exciting news for the future of BeeWare
The biggest challenge for any Open Source project is finding the resources needed to support development - and the BeeWare project is no exception.
To date, BeeWare has been primarily driven by the efforts of volunteers. We've been able to achieve amazing things with those volunteer efforts, demonstrating that Python is a viable language for GUI development on macOS, Windows, Linux, iOS and Android, as well as providing a proof-of-concept demonstration on Web. However, there are still lots of features that we'd like to add, and lots of ongoing maintenance that needs to be done as operating systems and hardware platforms evolve.
Over the years, we've been incredibly fortunate to have the financial support of our members. While I'm incredibly grateful for that support, it falls a long way short of even one full-time salary. We've also received grants (such as the PSF grant), and short-term contracts. These grants and contracts have enabled us to add specific (and much needed) features to BeeWare - but they don't provide resources for ongoing development and maintenance.
However, the brutal truth is that our rate of progress is severely limited by the volunteered resources at our disposal. As a result, I've spent a lot of time over the last few years thinking about how we can fund the work that needs to be done to make BeeWare the go-to solution for GUI development on desktop, mobile, and more - and do so without compromising the Open Source roots of the project.
I'm incredibly excited to announce that I've found a solution.
On March 21, I joined Anaconda as a Principal Engineer and Team Lead - and the team I will be leading will be focused 100% on BeeWare.
Anaconda is no stranger to Open Source. In addition to indirectly funding a number of community efforts through their Anaconda Dividend program, they directly employ people to work on Numba, Dask, Pyston and more. BeeWare will be joining this stable of projects.
Anaconda is supporting BeeWare in this way because they see the potential for Python as a language for GUI development - especially on mobile platforms. Python has established itself as an incredibly popular language in education, science, business and more - and in all those markets, there is potential to develop apps. By adding mobile platform support and app development to the list of tasks to which Python is already incredibly capable, Anaconda hopes to ensure the long term prospects of Python as a language, as well as ensuring a healthy market for the products they sell commercially.
BeeWare will continue to be a 100% Open Source project, with all contributions welcome from the community. The only difference is that Anaconda will be directly employing some of the contributors to the project. The conversations I've had with Anaconda's engineering leadership indicates they are acutely aware of the tension that exists between corporate funding and the technical direction of an Open Source project. They've indicated that they want to ensure that BeeWare's technical direction remains independent in both appearance and substance. This includes financial independence - the financial support provided by our members will remain vitally important, as it provides project-level funding for resources such as hosting and promotional materials.
The related good news is that part of my job title is "Team Lead" - so in the very near future, we'll be looking for people to join me on my team. I'm already working out the details of the team that I'd like to have, so watch this space for hiring announcements.
Anaconda has a long history of understanding and contributing to both the Python ecosystem and the Open Source community. Adding BeeWare to their stable of supported projects has enormous potential for BeeWare, and the Python ecosystem as a whole. I'm incredibly excited for what the future holds.
Some new ways to discuss BeeWare
Today, the BeeWare project is making 2 changes to the way we organize our community.
Firstly, we're going to move our real-time chat community from Gitter to Discord.
When we originally set up Gitter, it was a strong option for a chat community - especially one that was focussed on a software project, due to its tight integration with Github. However, over the years, Gitter has undergone multiple changes of ownership, but has remained essentially unchanged - in some cases, it's gotten appreciably worse (their native iOS and Android clients are effectively deprecated). In the meantime, Discord has emerged as a dominant player for community chat. It has a robust mobile client, and really good tools for community moderation, as well as the option of hosting video and voice chats - something we hope to be able to use for coordinating virtual sprints.
To join the new BeeWare Discord server, visit this link.
Secondly, we're going to start using Github Discussions.
For many years, we've tried to use Github Issues as a pseudo-forum. There's a lot of similarity between how a forum works and how Github displays and manages issues. Each "issue" is effectively a forum thread, and each comment is a message in that thread. However, there are also some important differences. Issues get closed when they're resolved; forum threads, however, don't necessarily have a resolution. Most importantly, "closing" an issue can look like someone is trying to kill discussion - when all they're really trying to do is keep the list of issues manageable.
The good news is that Github has noticed the similarity too, and has launched Github Discussions. Discussions offers a familiar "forum" interface, but with the familiar Github message interface, and tight integration with Github Issues and pull requests.
We've enabled Discussions on all the key BeeWare projects - Toga, Briefcase, Rubicon ObjC, Rubicon Java, Colosseum, Travertino and Podium.
Naturally, all of these new community spaces - along with all existing BeeWare spaces - are subject to the BeeWare Community Code of Conduct.
We hope these two changes will make it easier to participate in the BeeWare community. See you there!
Setzt die Android-Bienen frei!
Vor 9 Monaten haben wir angekündigt, dass das BeeWare-Projekt Empfänger eines PSF Education Grants sei. Ende 2019, haben wir Asheesh Laroia zur Umsetzung der Arbeiten gewählt. Heute können wir mit Stolz verkünden, dass die Android-Unterstützung von BeeWare einen Punkt erreicht hat, an dem sie für den Massenkonsum bereit ist.
Das BeeWare-Tutorial wurde aktualisiert und enthält nun auch eine Android-Spur. Das Tutorial führt dich durch den Prozess, deine erste Anwendung zu schreiben und zu deployen - zuerst als Desktop-Anwendung, dann auf deinem Smartphone. Alles, was du dazu brauchst, ist eine Python-Installation; alles andere wird automatisch von den BeeWare-Tools als Teil des Entwicklungsprozesses heruntergeladen.
Als Beweis, dass das Ganze wirklich funktioniert, haben wir Travel Tips im Google Play Store hochgeladen. Im Google Play Store wurde somit eine Python-Anwendung deployed, die aus demselben Quellcode besteht wie die Version im iOS App Store.
Asheesh gebührt großer Dank für all die Arbeit, die er in dieses Projekt investiert hat. Die Bereitstellung des Android-Supports in BeeWare hat beträchtliche technische Fähigkeiten, Liebe zum Detail und kreative Hacks erfordert; und Asheesh hat diese Arbeit konsequent geleistet. Ein großer Dank geht auch an das Education Grants Committee der Python Software Foundation für das Vertrauen, das sie in BeeWare gesetzt haben, als sie dieses Projekt finanziert haben. Innerhalb von 6 Monaten (während wir gleichzeitig eine Weltpandemie zu bekämpfen hatten), sind wir von einem Nichts zu einer vollständigen Android-Implementierung gekommen. Ohne ihre finanzielle Unterstützung wäre dieses Projekt immer noch ein Traum.
Es gibt aber noch viel zu tun. Es gibt immer noch eine Menge Widgets, die Android-Implementierungen benötigen, und neue Widgets, die wir hinzufügen wollen. Wir möchten Unterstützung für Gerätefunktionen wie Kameras, GPS und Beschleunigungsmesser hinzufügen. Wir möchten die Verwendung von binären Python-Paketen wie NumPy, Keras und mehr erleichtern. Und wir möchten die Arbeit, die wir geleistet haben, in CPython selbst zusammenführen.
All dies sind an sich schon bedeutende Projekte, die einen vergleichbaren Aufwand erfordern, wie Android als unterstützte Plattform hinzuzufügen. Und deshalb suchen wir nach finanzieller Unterstützung, um dies zu ermöglichen. Wir bewerben uns um neue Zuschüsse, wenn sich neue Möglichkeiten ergeben, aber die hilfreichste Finanzierungsquelle sind die laufenden Mittel, die sich aus Mitgliedschaften ergeben. Laufende Finanzierung bedeutet, dass wir uns auf die Verbesserung von BeeWare konzentrieren können, anstatt uns um Zuschüsse zu bemühen. Es bedeutet, dass wir in der Lage sind, permanentes Personal einzustellen, anstatt kurzfristige Verträge anzubieten. Und es bedeutet, dass wir in der Lage sind, langfristige Pläne und Versprechungen gegenüber der Gemeinschaft zu machen.
Wenn dieses Projekt etwas beweist, dann, dass Geld etwas bewegt. Wenn du von den Aussichten von Python auf mobilen Plattformen begeistert bist, ziehe bitte in Betracht, dem BeeWare-Projekt als finanzielles Mitglied beizutreten. Falls du Erfahrung mit der Kommerzialisierung von Open-Source-Projekten oder weitere Ideen hast, bitte nimm mit uns Kontakt auf.
Python auf Android: Es lebt!
In den letzten Monaten hatten wir einen Auftragnehmer, (Asheesh Laroia) der daran gearbeitet hat, die Android Unterstützung für die BeeWare Werkzeugsammlung wiederherzustellen.
Ich bin sehr froh mitteilen zu können, dass wir unseren ersten großen Meilenstein erreicht haben: Eine funktionierende, reine Python App, die auf einem Android Gerät läuft!
Das ist nicht das Ende des Projekts - es ist erst der Anfang. Es gibt immer noch viel Feintuning zu machen (Insbesondere was die Größe der Support Bibliotheken betrifft), und wir müssen diese Unterstützung in Briefcase und Toga integrieren.
Trotzdem können Abenteuerlustige Asheeshs work-in-progress Entwicklung ausprobieren. Seine Python Android Support repository enthält den aktuellen Stand der Entwicklungen, und enthält ziemlich gute Anweisungen um loszulegen. Momentan benötigt man noch zumindest ein bisschen Wissen über native Android Programmierung um dieses Repository voll nutzen zu können; aber wenn man die Resultate aus dem Video wiederholen möchte, sollte dieses Repository (und die darin verlinkten Repositorys) alles bereitstellen was man braucht.
Ein großes Dankeschön geht mal wieder an die Python Software Foundation. Ohne ihre finanzielle Unterstützung, wäre diese Arbeit immer noch nur ein Konzept. Dieses Projekt ist nur eine von vielen Arten, auf die PSF Nutzer Spenden die Python Gemeinschaft und das Python Ökosystem untertützen. Wenn Ihre Firma Python in irgendeiner Art und Weise nutzt, empfehle ich Ihnen sehr die PSF finanziell zu unterstützen, damit sie Projekte wie dieses weiterhin unterstützen kann.
Ein großes Dankeschön gebührt auch Asheesh. Ohne seine herrausragenden Fähigkeiten, Liebe zum Detail, und seine Begeisterung für Compilerfehler, hätten wir nicht den unglaublich schnellen Fortschritt erzielt, den wir jetzt beobachten können.
Freut euch bald auf weitere Ankündigungen!
Wir haben einen Auftragnehmer für unseren Android-Vertrag!
Vor einigen Monaten haben wir angekündigt, dass das BeeWare-Projekt von der PSF einen Zuschuss erhalten hat, um unsere Unterstützung für Android zu verbessern. Damals riefen wir Auftragnehmer auf, uns bei der Durchführung dieser Arbeit zu unterstützen.
Wir freuen uns sehr, euch mitteilen zu können, dass wir jetzt einen Auftragnehmer ausgewählt haben: Asheesh Laroia.
Asheesh ist ein regelmäßiger Referent bei Python-Veranstaltungen, wo er sich in eine Reihe von detaillierten und komplexen Themen vertieft hat. Er hat uns auch mit der Liste unkonventioneller technischer Integrationsprojekte beeindruckt, an denen er in professioneller und gelegentlicher Tätigkeit mitgewirkt hat.
Auf die Frage, warum er sich für eine Zusammenarbeit mit BeeWare für diesen Auftrag beworben hat, sagte Asheesh: "Ich benutze jeden Tag ein Android-Telefon und fühle mich geehrt, an der Umsetzung der BeeWare-Vision mitwirken zu können, Python zur Erstellung erstklassiger, nativer Anwendungen einzusetzen.
Asheesh wird seine Arbeit Mitte Dezember aufnehmen, und wenn alles gut läuft, sollten wir ab Mitte bis Ende Februar signifikante Ergebnisse sehen. Wenn du den Fortschritt verfolgen möchtest, kannst du BeeWare auf Twitter verfolgen; wir werden auch größere Updates auf diesem Blog veröffentlichen.
BeeWare-Projekt erhält PSF-Bildungszuschuss
Das BeeWare-Projekt will es allen Python-Entwicklern ermöglichen, native Anwendungen für Desktop- und mobile Plattformen zu schreiben. Wir haben solide Unterstützung für die meisten Desktop-Betriebssysteme und iOS, aber wir wissen, dass unsere Android-Unterstützung fehlt. Das BeeWare-Kernteam weiß, was getan werden muss, um das Problem zu lösen - was uns bisher gefehlt hat, sind Zeit und Ressourcen.
Dank der PSF-Bildungszuschussgruppe ist das kein Thema mehr. Wir haben einen Zuschuss von 50'000 US-Dollar erhalten, um die Android-Unterstützung von BeeWare auf ein Niveau zu bringen, das mit unserer iOS-Unterstützung vergleichbar ist. Da wir derzeit nicht die Zeit haben, die Arbeit selbst zu erledigen, rufen wir Auftragnehmer auf, uns bei der Bereitstellung dieser Unterstützung zu helfen.
Es handelt sich um einen bezahlten Vertrag, der voraussichtlich 3-6 Monate dauern wird (abhängig von der Erfahrung des gewählten Auftragnehmers). Du musst auch nicht in den USA oder Europa ansässig sein; die Möglichkeit steht allen offen, die die Anforderungen des Vertrags erfüllen.
Leider erfordert diese Aufgabe einige anspruchsvolle Fähigkeiten, und wir sind nicht in der Lage, eine umfassende Betreuung anzubieten. Für eine erfolgreiche Bewerbung werden voraussichtlich einige Erfahrungen und eine Vorgeschichte mit den betreffenden Technologien erforderlich sein.
Eine vollständige Rollenbeschreibung und Arbeitsumfang für den Vertrag ist verfügbar. Um dein Interesse zu bekunden, sende bitte deinen Lebenslauf und dein Bewerbungsschreiben an contracts@beeware.org.
Wir freuen uns darauf, in naher Zukunft volle Unterstützung für Android ankündigen zu können!
2018 Google Summer of Code - VOC Optimization
Google Summer of Code is coming to an end. I've spent the summer working on optimizing the VOC compiler, and I’m super excited to share the results.
Results
There are a couple of ways to evaluate the performance improvement from my project.
Microbenchmarks
Firstly, we introduced a microbenchmarking suite. Each microbenchmark is a small piece of Python code that tests a single and specific Python construct, or datatype, or control flow. The benchmarking infrastructure itself is crude (essentially it just tells you the total amount of processor time it took to run, with no fancy statistics) but it has been extremely useful to me while working on performance features to verify performance gain.
The idea is that the benchmarking suite is not to be run as part of the full test suite, but rather as needed and manually whenever an optimization is implemented. It also provides a way to check and prevent performance regression, especially on the "optimized" parts of VOC. While it doesn't really make sense to record specific numbers, as they will always vary from machine to machine, it should be reasonably easy to compare two versions of VOC. Benchmark numbers are included on each optimization-related PR I've worked on this summer (see PR log below), and I hope that more benchmarks will be added as more performance efforts are carried out in the future.
Pystone
Pystone is a Python Dhrystone, a standard benchmark for testing the performance of Python on a machine. Here are the before and after results on my machine:
May 10th, 2018:
$ python setup.py test -s tests.test_pystone test_pystone (tests.test_pystone.PystoneTest) ... Pystone(1.2) time for 50000 passes = 101.833 This machine benchmarks at 490.998 pystones/second
$ python setup.py test -s tests.test_pystone test_pystone (tests.test_pystone.PystoneTest) ... Pystone(1.2) time for 50000 passes = 101.298 This machine benchmarks at 493.595 pystones/second
$ python setup.py test -s tests.test_pystone test_pystone (tests.test_pystone.PystoneTest) ... Pystone(1.2) time for 50000 passes = 102.247 This machine benchmarks at 489.014 pystones/second
On current master (Aug 14th, 2018):
$ python setup.py test -s tests.test_pystone test_pystone (tests.test_pystone.PystoneTest) ... Pystone(1.2) time for 50000 passes = 11.2300 This machine benchmarks at 4452.37 pystones/second
$ python setup.py test -s tests.test_pystone test_pystone (tests.test_pystone.PystoneTest) ... Pystone(1.2) time for 50000 passes = 10.9833 This machine benchmarks at 4552.36 pystones/second
$ python setup.py test -s tests.test_pystone pystone (tests.test_pystone.PystoneTest) ... Pystone(1.2) time for 50000 passes = 10.9498 This machine benchmarks at 4566.29 pystones/second
Conclusions
Some things that I learned about VOC while working on this project:
1. Object creation in the JVM is expensive. This definitely does not mean that the VOC user writing Python should think about minimizing the number of objects that she creates, but rather that any time we can non-trivially reduce the number of objects created during bytecode transpilation or in VOC-defined function calls, we can expect to see a huge performance boost. Integer and boolean preallocation, which is about reusing objects that have already been created, was one of the most significant improvements we made this summer.
2. Method calls in VOC are expensive. This is essentially due to the process of invoking a callable: you have to check that the method is defined on the object, then construct it (read: object creation!), and check the arguments, before it can actually be called. (This is done using reflection, which is super interesting and confusing in itself.) And this is the reason why refactoring the Python comparison functions made such a big performance impact, because we were able to circumvent this process.
3. Exception-heavy code is expensive. Again, this is not to say that the programmer is on the hook for being frugal when throwing exceptions, but that VOC benefits greatly by avoiding the use of exceptions internally except when strictly necessary. For instance, Python uses StopIteration exceptions to signal the end of a for loop, and they quickly rack up when you have nested loops (everything is ultimately related to object creation!). That was the motivation for the nested loops optimization.
If I may be a bit more reflective here, one of the a-ha! moments I had this summer was realizing that to really optimize something, you have to understand where its biggest problems are first. I remember pitching to Russ at the start of the summer things like loop unrolling, constant folding, even converting to SSA-form (you know, stuff I heard about optimzation in my compilers class) and he was saying to me, think simpler. While working on my project, I used a profiler to understand exactly which parts of VOC were slow, and that information drove the changes we implemented. I think it worked out pretty well!
Future Work
- Minimize boxing of primitive types like String and Int. As VOC is written half in Python, half in Java, a single integer can be found in various representations on its way through the compiler -- as a Python object, unboxed to a primitive Java int, then packaged back up to a Python object. This problem was (somewhat incoherently) addressed in my proposal, but ultimately we couldn't come up with a good abstraction to support it.
- Build a peephole optimizer. CPython's peephole optimizer scans generated bytecode to identify sequences of bytecode that can be optimized, VOC could benefit from this too.
- Hook up more benchmarks, which serve as both proof of the kinds of programs VOC can currently compile and areas ripe for performance improvement.
Thank you
I will wrap this up by giving big thanks to Russ, my mentor. The time you spent helping me form my ideas, patiently answering my questions and reviewing my work was invaluable to me. It couldn't have been easy keeping up with what I was doing especially since I started improvising halfway through the summer. I am so grateful for your help, thank you.
PR Log (in chronological order)
Bug Fixes and Miscellaneous
- Fix Method repr
- Fix custom substitutions
- Fix List Bug
- Remove Unnecessary Instruction
- Fix __setitem__ error messages
- Fix contains/not contains bugs and refactor
- Add tests for problematic exception raising
- Add test for with + exception combo
- Add test for wrong iter error message
- Add tests for globals bug
- Remove unnecessary type casts and clean up
- Add test for problematic builtin function call
- Introduce org/python/Object type tests
2018 Google Summer of Code - Implement asyncio support in VOC
In the blink of an eye, Google Summer of Code (GSoC) 2018 has come to an end. During the three months long coding period, I have contributed several patches in VOC repository of BeeWare, all working towards the ultimate end goal of running asyncio module in VOC. In this blog post (which is my first actual blog post by the way 😄), I will document what I have done so far, why I couldn't make it to the end goal (yea, unfortunately I couldn't get asyncio to work at the end of GSoC 2018), and what's left that needs to be done in order to achieve the end goal (or at least make part of asyncio work).
Building Foundation
The first error that the transpiler throws when attempting to compile asyncio module was "No handler for YieldFrom", so it makes sense to start from this issue first.
Another feature related to generator was Yield expression. Before GSoC 2018, Yield statement in VOC was just a statement, meaning yield could not be used as expression. Generator methods such as generator.send, generator.throw and generator.close were not supported as well. Those features are what make asynchronous programming with generator possible, so I spent a few weeks to extend generator functionality in VOC, laying down the path to asyncio module.
PRs related to generator are listed below:
Nonlocal Statement
Nonlocal statement was another syntax not supported by VOC. After completion of generator's features, implementing this is the next step towards compiling asyncio module.
Implementing this feature took about 3 ~ 4 weeks as this is not as trivial as it seems. I took several approaches on this, while some of them do work, the code is not pretty and hacky, which could come back to bite me/other contributors in the long run. After many discussions with Russell, I refactored the closure mechanism in VOC and took a much cleaner approach in nonlocal implementations. I must admit that I took some short-cuts for the sake of "making nonlocal works" in the process of implementing nonlocal statement, resulting in poor design and messy codes. Many thanks to Russell, who helped me to improve my coding style and told me not to be discouraged when I'm stuck. 😄
Related PRs:
The Collections Module
Next item on my hit list was pure Java implementations of the collections module. asyncio module depends on 3 data structures from collections, namely defauldict, Deque and OrderedDict. Two of them (defaultdict and Deque) are implemented in C in CPython, plus they have good analog in Java, so it makes senses to implement the module in Java. Porting defauldict, Deque and OrderedDict to Java in VOC is relatively straight-forward, taking about 1.5 weeks to complete.
Related PRs:
Other PRs submitted during GSoC 2018
- PR #817 : Added coroutine related exception class [WIP] (closed due to not needed)
- PR #836 : Changed Bool construction to use getBool instead (merged)
- PR #847 : Add custom exceptions test cases (closed due to more comprehensive handling in PR #844)
- PR #849 : Fixed Unknown constant type <class 'frozenset'> in function definition (merged)
- PR #858 : Added test case for Issue #857 (merged)
- PR #860 : Added test case for Issue #859 (merged)
- PR #862 : Added test case for Issue #861 (merged)
- PR #867 : Fixed Issue #866 RunTimeError when generator is nested in more than 1 level of function definition (merged)
- PR #868 : Fixed Issue #861 Redefining nested function from other function overrides original nested function (merged)
- PR #879 : Fixed Incompatible Stack Height caused by expression statement (merged)
- PR #901 : Added test case for Issue #900 (merged)
- PR #788 : Implements asyncio.coroutines [WIP] (open, the dream 😎)
Issues submitted during GSoC 2018
- Issue #861 : Redefining nested function from other function overrides original nested function (fixed in PR #868)
- Issue #866 : RunTimeError when generator is nested in more than 1 level of function definition (fixed in PR #867)
- Issue #828 : Finally block of generator is not executed during garbage collection (open)
- Issue #857 : Complex datatype in set cause java.lang.StackOverflowError (open)
- Issue #859 : Duplicated values of equivalent but different data types in set (open)
- Issue #900 : Exception in nested try-catch suite is 'leaked' to another enclosing try-catch suite (open)
- Issue #827 : Maps reserved Java keywords to Python built-in function/method call (closed)
Towards The Ultimate End Goal
Unfortunately, three months of GSoC coding period was not enough for me to bring asyncio module to VOC. The nonlocal statement implementation was the biggest blocker for me mainly because I didn't think thoroughly before writing code. If I were to plan carefully and lay out a general coding direction, I would've completed it in much shorter time and have time for other implementations. An advice for the aspiring and upcoming GSoC-er, don't rush your code, make sure you know 100% about what you're doing before diving into the codes.
With that said, following are the list of modules to be implemented/ported to Java before asyncio will work in VOC:
- socket module (a bit tricky since Java doesn't support Unix domain socket natively)
- selectors module (high level I/O operations)
- threading module (might be easier to implement this first since threading in Python is an emulation of Java's Thread)
- time module (partially implemented in VOC)
Final Thoughts
Huge thanks to my mentor, Russell Keith-Magee for accepting my proposal, providing guidance and encouraging me when things didn't go as intended. It is truly an honor to be a part of the BeeWare community. I had a blast contributing to BeeWare project, and I'm sure I will stick around as a regular contributor. Also shout out to the BeeWare community for answering my queries and reviewing my pull requests. 😄