Backlog

Planned

In Progress

Completed

General roadmap

With this feature, you will be able to add loops inside your tests

In Review

Loop / while inside tests

Paweł Bylina

Dark theme of the BugBug web application.

Dark mode

Please add the functionality to configure browser geolocation to a specific coordinate/region for a test/suite.This feature would be crucial to test many apps since their behaviour might be adjusted in accordance to the geolocation, and in many cases this is a regulatory requirement of certain countries/industries.

Configure browser geolocation

Michał Muszyński

Folder system for components for better organization. We may use some components for a particular section of our product and some others for another section. It could be useful to have a way of distinguishing them.

Folder system for components

Ioannis Psaronikolakis

Multi-browser support

Today, BugBug offers the only option to configure Desktop and Mobile window sizes.It would be nice to give the ability to add more screen resolutions.<img src="https://fb-usercontent.fra1.cdn.digitaloceanspaces.com/1710334918719..png">

Option to add more window sizes for tests in project settings

This will only be available for cloud runs, but could provide some additional information for debugging purposes.

Video of a test run

A new type of step that must be confirmed whether passes or fails manually by the user.<img src="https://fb-usercontent.fra1.cdn.digitaloceanspaces.com/1706715232343..png">We are gathering interest in this idea here.The step can be used to create a fully manual test scenario, where all steps are to be verified by the QA professional or can be used inside a regular automated test where at the certain stage the manual verification by the QA professional is necessary (e.g. is the content of the generated PDF valid).EXAMPLE OF MIXED-CASE SCENARIOPhase 1 — test recording<ul><li>Marie starts test recording, logs into the application and navigates to analytics dashboard.</li><li>She selects the first position on the list of available metics.</li><li>She records a series of assertion ensuring proper graph headers are present on the page.</li><li>Next, she pauses the test recording and validates if all the graphs on the page contain no data gaps. She pans and zoom inside the interactive canvas-based visualization.</li><li>After confirming no data gaps are present in the graphs, she resumes the test recording.</li><li>She selects another position on the list of available metics, records a series of assertion ensuring proper graph headers are present on the page.</li><li>Next, she pauses the test recording once again and validates if all the graphs on the page contain no data gaps. Once again, she pans and zoom inside the interactive canvas-based visualization.</li><li>After confirming no data gaps are present in the graphs, she finishes the test.</li></ul>Phase 2 — test editing<ul><li>Now, Marie goes back to the BugBug web application to split the test into several groups and to create placeholder groups for the manual verification process for both metrics.</li><li>She adds a new manual step in the manual verification group and fills out the name: "Check no data gaps in YOY revenue graph" and provides a quick description in the step details. She repeats the process for the other metric.</li></ul>Phase 3 – test run<ul><li>Marie runs the test. BugBug extension goes through the previously steps and stops at the manual step "Check no data gaps in YOY revenue graph".</li><li>She validates that no gaps are present and sets the step toggle into "Passed" position.</li><li>The extension immediatly resumes replaying previously recorded step until it faces the manunal step again.</li><li>Marie repeats the procedure and after confirming the step passed, the whole test finishes with "Passed" status.</li></ul>Technical requirements<ol><li>step name is provided by the user</li><li>additional description can be provided in the step details</li><li>step timeout can be set by the user</li><li>step is set to passed in the cloud environment</li><li>if run via BugBug extension the popup lets user confirm if the step passed or failed</li></ol>

Manual step in tests

Some components should only be editable by an admin user, the normal users should only be able to use the components but should not modify them

Lock components for non-admin users

Tobias Schittkowski

Can we have a performance / load test feature in BugBug, to ensure that our application will meet the service levels expected in production as well as to deliver a positive user experience. With the option to choose how many virtual users should be running and to generate a performance report.It would also be great to have a graph/raw data that shows how a single test perform over time, to see the if the test is getting slower or not.

Performance / load test

Christian Wethje

As you create more tests it would be harder to filter and organize the scripts, would be nice if we can create a folder inside to organize it.<div data-type="horizontalRule"><hr></div>Author: Lain HA nice feature would be the ability to group tests in the same way test steps can be grouped and the group named, expanded and collapsed. So, allow tests to be grouped, the ground named, and the individual tests expanded or collapsed in a similar fashion. A step further would be to allow a test to appear in more than one group, but I guess that’s then crossing over into Suites. Or maybe tests could be tagged and have more than one tag, and then viewed by tab with tabs expanded and collapsed? That would do 2-in-1 - grouping and allowing one test to be seen under more than one topic/slice (tag)

Add a way to create a folder in tests page

Rav Banaag

Hey there,To integrate Bugbug into my CI/CD and have my tests pass on multiple environments, I currently have to duplicate the tests and modify the changed parts, which is inefficient...It would be great if we could have test versioning so that I could point the production-oriented test to the production environment (let's name it v1.0) and the development-oriented test (v1.1) to my development environment.With that, when I deploy dev to prod, I'll be able to point the test I've already created for dev (v1.1) directly to production.Without that, I'll have to spend time modifying the tests right after&nbsp;deploying dev to production, otherwise the tests would fail.

Add a versioning system to allow running an old version of the test

Tegomass

In some test cases, to cover scenarios, retrieving data from API is required. Today, this is something that users can achieve using a custom JS step and fetch() function, but this requires code.To simplify this process, it would be nice to add an extra step like Request API where you will be able to define a method, URL, and parameters for making requests. Using variables within this step should also be possible. Storing the response as a variable could also give a lot of flexibility to users.This could be a first step to implementing more advanced API testing for BugBug.

Easy API testing

An easy way to validate what's in the generated PDF

PDF assertions

Two-factor authentication (2FA) for BugBug web application

We plan to add an option to perform some actions before test execution and (always) after test execution.This could be treated in a similar way to the setUp and tearDown functions in unit tests.

Before & After steps for tests

I want to be able to tag tests and to filter and sort by tags, results, last execution time, etc.

Adding a tag to a test

Bartosz Wachowski

I've had this happen a few times when I wanted to delete a test step but didn't want to finally. It would be great if we can undo (at least) the last delete action. Using Ctrl+Z or a cancel deletion button somewhere.

Allow undoing a test step deletion

Example:<ul><li>user has a list of company IDs that he wants to use in tests</li><li>each test execution should take the next ID from the list</li></ul>BugBug could help here if there is an option to somehow store information about the last used ID. This could also be used in other cases, or for exchanging data between tests in the test suite.

Storing information between tests / persistent variables (data storage)

Add a native integration with Teams, similar to Slack.

Teams integration for alerts

BugBug does not currently have an import/export feature for tests. Exporting and importing tests can be very beneficial for users. By exporting tests, users can save their work and reuse it in other frameworks. This allows users to benefit from further advanced testing without losing their work. Users can also export multiple tests into a single file, which is compatible with many other frameworks (Selenium, Python). The export feature is especially useful to outgrow the capabilities of BugBug. In addition, users can import tests from other frameworks into BugBug (Selenium, Python). This allows users to reuse their existing test cases and test suites in BugBug. Users can also import multiple tests into BugBug at once, which saves time and effort.

Attualmente BugBug non dispone di una funzione di importazione/esportazione dei test. L'esportazione e l'importazione dei test possono essere molto utili per gli utenti. Esportando i test, gli utenti possono salvare il proprio lavoro e riutilizzarlo in altri framework. Ciò consente agli utenti di beneficiare di test più avanzati senza perdere il proprio lavoro. Gli utenti possono anche esportare più test in un unico file, compatibile con molti altri framework (Selenium, Python). La funzione di esportazione è particolarmente utile per superare i limiti di BugBug. Inoltre, gli utenti possono importare test da altri framework in BugBug (Selenium, Python). Ciò consente agli utenti di riutilizzare i propri casi di test e suite di test esistenti in BugBug. Gli utenti possono anche importare più test in BugBug contemporaneamente, risparmiando tempo e fatica.

Funzionalità di importazione/esportazione dei test

Import/Export feature for tests

Haian Abou-Karam

ShadowDOM is a "new" way to handle encapsulation in web applications <a target="_blank" rel="noopener noreferrer nofollow" class="text-indigo-500 dark:text-indigo-300 cursor-pointer" href="https://developer.mozilla.org/en-US/docs/Web/API/Web_components/Using_shadow_DOM">https://developer.mozilla.org/en-US/docs/Web/API/Web_components/Using_shadow_DOM</a>It is also a challenge for test automation. BugBug doesn't support this technology today. We plan to add support for capturing and executing all interactions within ShadowDOM elements.

ShadowDOMは、ウェブアプリケーションのカプセル化を扱う「新しい」方法である<a target="_blank" rel="noopener noreferrer nofollow" class="text-indigo-500 dark:text-indigo-300 cursor-pointer" href="https://developer.mozilla.org/en-US/docs/Web/API/Web_components/Using_shadow_DOM">https://developer.mozilla.org/en-US/docs/Web/API/Web_components/Using_shadow_DOM</a>また、テスト自動化の課題でもあります。 BugBug は現在この技術をサポートしていません。ShadowDOM 要素内のすべてのインタラクションをキャプチャし、実行するためのサポートを追加する予定です。

ShadowDOMのサポート

High Priority

Support for ShadowDOM

For better test diagnosis, we plan to collect all browser network logs and present them with the test run.Related ticket <a target="_blank" rel="noopener noreferrer nofollow" class="text-indigo-500 dark:text-indigo-300 cursor-pointer" href="https://feedback.bugbug.io/p/collect-browser-console-logs-for-each-test-run">https://feedback.bugbug.io/p/collect-browser-console-logs-for-each-test-run</a>

Collect browser network logs for every test run

This feature could be useful for no-code app builders or even Vercel builds.

Incoming webhooks for launching tests and suties

In some applications, <code>drag and drop</code> functionality is critical. BugBug supports some of these actions, but we know that it has a lot of trouble with more complex scenarios.The idea here is to rewrite this feature from scratch to support all kinds of drag &amp; drop interactions.

Better drag & drop support

'If-else' rules apply to tests for single steps and whole groups.

テストにおける「if else」のルール

簡単な条件文

Simple conditional statements

We plan to add a separate step for screenshot comparison. The first execution will save a screenshot as a baseline, and then every next execution will be compared to this baseline screenshot. If the new screenshot is different from the baseline, the test will fail.Possibly, this step should also have an option to<ul><li>exclude some random elements from the DOM before taking a screenshot</li><li>option to decide how this comparison should be done (pixel perfect or percentage)</li><li>option to verify screenshot only for specific element on the page (given selector)</li></ul>Any other ideas around this feature are more than welcome.

Wir planen, einen separaten Schritt für den Screenshot-Vergleich hinzuzufügen. Bei der ersten Ausführung wird ein Screenshot als Basislinie gespeichert, und jede weitere Ausführung wird mit diesem Basis-Screenshot verglichen. Weicht der neue Screenshot von der Baseline ab, wird der Test fehlschlagen.Möglicherweise sollte dieser Schritt auch eine Option enthalten, um<ul><li>einige zufällige Elemente aus dem DOM auszuschließen, bevor ein Screenshot erstellt wird</li><li>die Option, zu entscheiden, wie dieser Vergleich durchgeführt werden soll (pixelgenau oder prozentual)</li><li>die Option, den Screenshot nur für ein bestimmtes Element auf der Seite zu überprüfen (mit einem bestimmten Selektor)</li></ul>Weitere Ideen rund um diese Funktion sind mehr als willkommen.

Screenshot-Vergleich (Visuelle Regression) - Durchsetzungsschritt

Screenshot comparison (Visual regression) - assertion step

Ability to add the same component to the test multiple times

For evidentiary purposes as defined by some legal regulations (in the USA), we plan to add an option to save the results of test runs and full suite runs as a PDF with detailed information such as<ul><li>Step type</li><li>Screenshot of the page</li><li>Additional comment</li><li>Step status</li><li>Date of the execution</li></ul>The first implementation should allow users to download these reports from the UI manually. Based on the feedback, the next phase could bring some automation on how to store them without manual interactions.Any extra feature requests are welcome.

More detailed PDF reports for test runs and suite runs

For me, the most time-consuming part after (and during) recording a test is the need to run the entire test from the beginning to the end (or until a certain point) if I need to change or update something.More exactly, for our longer E2E tests from start to finish they can take up to 5+ minutes, and if I need to record something at the last few groups, I need to wait minutes in a row just to reach that part in order to click or update some new elements.In the <a target="_blank" rel="noopener noreferrer nofollow" class="text-indigo-500 dark:text-indigo-300 cursor-pointer" href="https://bugbug.io/roadmap/">roadmap</a>, something that would help with the above are the following two considered features:<img src="https://fb-usercontent.fra1.cdn.digitaloceanspaces.com/1700492597656.png">I wish that, during recording, I could "select" specific locations inside the test where the step would appear for the action I do in the recorded window. This way I could skip/ignore certain groups/components of the tests which I know are passing.Maybe something like a "Record at location" or "Change record location" button in the recording options modal, which, when pressed, would allow me to select where my action will be recorded in the test in the form of a step.If I was able to do something similar to that, I would be at least 50% faster if not more, while expanding our testsuite or even keeping it updated when needed. As mentioned, a big chunk of my time consists in waiting for a test to reach a certain point, so I can continue adding more steps.I think the best parallel I can do here is the "Record at cursor" option from Playwright:<ul><li>While this option is activated (and clearly marked it's on) then any action done on the webpage will be recorded</li><li>I can cancel/stop/pause the option, without closing the recording window, so that I can freely navigate the webpage elements without actually recording steps.</li><li>While the option is activated, I can also click any line number in the code editor where I want the step to be recorded</li><li>The steps are recorded in the same manner as BugBug, from top to bottom, wherever the "cursor" is at the code line at that moment</li></ul>A video that captures the above behavior to get a better idea:<a target="_blank" rel="noopener noreferrer nofollow" class="text-indigo-500 dark:text-indigo-300 cursor-pointer" href="https://streamable.com/w18bz1">https://streamable.com/w18bz1</a>In my opinion, this would be the most important feature for me while using BugBug, probably above any other improvements I can think of. 😁

Record (and run) at a specific location in a test

Marius COJOCARIU

It could be very useful to be able to sort every column in a test report as well as having filters to show at least the passed/failed/all tests. You can also, decouple the test code from the test name in order for that to be individually sorted as well.

Sorting and filtering on test suite report

The idea is to be able to assign members of my organization to one or multiple project.The goal is to avoid a developer that work only on the project A to have an access to project B.

Organization - Assign users to projects

An easy way to add data about failed tests and step directly to Jira as a ticket.

Jira integration

On default, this is set up to be UK. I would like an option to change it either in the project settings or after contacting support.

Changing location on cloud runs

On some web pages, when I click "Add assert" in the BugBug overlay during the recording session, the current element of the page loses focus and hides the page fragment I want to record.As a possible solution, BugBug could handle a keyboard shortcut to switch assertion mode, which will keep the focus on the current element page and allow the recording of necessary elements.