Recently, we launched a major update that integrates WordPress directly into the CodeWP platform. Users who create snippets and plugins can now easily push them onto a live version of WordPress core running directly in their browser. They can not only manually preview and test but also utilize CodeWP to automatically retrieve error logs and revise code if necessary (beta testing).
The Idea
Our WordPress-specific AI is efficient, yet the inherent limitations of LLMs mean that code generation won't always be perfect. Sometimes the code may include deprecated functions or code that simply doesn't work. This is part of the process, and traditionally, our users would test their code on a sandbox website like InstaWP. After iterating, they would then install it on staging and production, following the standard WordPress testing workflow.
Even before AI, this was the general testing workflow if you wrote code. It involved a lot of back-and-forth between IDEs, multiple WordPress environments, code snippet plugins, etc.
When you make a change, it needs to be tested, so you copy it into a code snippet plugin. If an error pops up, you go back to your code editor, make the necessary changes, and then return to WordPress to test again.
Our users have always sought an easier way to test WordPress code, and since the beginning, we've experimented with several methods to facilitate this. This includes easily downloading PHP files, exporting packages as plugins in version 2.0+, and alpha features of our helper plugin that allow direct code sending to a website.
However, what we desired was a loop. A way to easily send code to WordPress with relevant plugins installed, effortlessly pull out any PHP warnings or errors, and automatically resolve them with our AI. This process would apply to each change made with our AI.
While this was possible with the aforementioned concepts, it was also quite complex in practice. They required extensive setup, weren't fast, and weren't directly integrated with the platform. But then, WP Playground came along...
The Implementation
With WordPress Playground, this "testing loop" has become a reality. A fully functional version of WordPress is embedded directly into the browser, accessible to both the user and our AI.
After a user generates a snippet or a multi-file plugin with our platform, they can easily test this code. All they need to do is click the big green testing button, and the platform springs into action. A customized instance of WordPress Playground is spun up, running WordPress directly in the user's browser for speed, security, and complete integration with our platform.
This instance automatically includes our helper plugin, along with any free plugins related to the code being generated. Currently, we don't support pro plugins for pro AI modes. For optimal results, users will need to manually install versions of these plugins on the instance themselves.
Our helper plugin serves a dual purpose. Firstly, it receives the code being tested and runs it on the WordPress website. Secondly, it monitors the environment for any errors and relays them back to our AI. This process completes in seconds. If any errors are reported, the AI suggests solutions for resolution. Once the user accepts these suggestions, the errors are fixed.
The user can continually edit the code – after each update, it's sent to the testing environment, and any errors are extracted. This process repeats until the user is satisfied with the implementation.
At the end of the session, the instance is completely destroyed, ensuring that no data is saved. Since everything runs directly in the browser and we've disabled network connections, privacy is preserved.
Why Care
We are a WordPress-specific platform, and this is one of the most industry-focused features we've added to date. You won't find any other AI tool that provides the most accurate generations for WordPress and surrounding plugins, all incorporating a complete version of WordPress directly in the application for testing.
Hallucinations are probably the largest issue when it comes to AI-generated code. As mentioned, due to the technical way LLMs work, it's difficult to completely eliminate hallucinations. Just as a normal developer can make mistakes with their code, so can AI. However, with normal development these mistakes are caught and revised during the testing phase.
We've taken the first step of adding a testing environment and added it directly to the platform giving users an automatic way to verify that code will work on bare WordPress, providing a very easy way to catch nonexistent functions, syntax errors, and more. For those who are technical, they can manually test various cases themselves. For those who are not, they can automatically use CodeWP to identify if there are errors, and how to solve them if they exist.
We're excited to continue improving this feature. In the future, we will expand the automated testing to cover all cases in the code. For example, if your code removes a WooCommerce checkout field, the AI will automatically navigate to the WooCommerce checkout, complete it, and ensure that everything is working as expected.
Read more: https://intercom.help/wpai/en/articles/8938126-preview-and-test-code
