Frequently Asked Questions
Yes, QMFactory has been developed from the ground up to support agile Test Management and test delivery within agile projects. Enginuity and Authorise AI automation products are designed to automate the test delivery process and execute limitless numbers of tests within hours via scalable virtual testers. All products use open API integration and can be fully integrated within Pinnacle, open source or other vendor products.
Yes, Pinnacle tools have been designed to support different project delivery options and can support a range to project methodologies, size and complexities.
Pinnacle’s AI automation supports all test types, against any technologies across the technology stack. Therefore, you can run tests within the UI, against API’s, interfaces, services and message layers, or against the backend data bases allowing from complete and comprehensive test coverage for your projects.
Yes, Pinnacle’s Enginuity can be used as an advanced no code AI automation framework over the top of Selenium for web and mobile applications. Enginuity is also able to convert Selenium’s labour intensive complex coded scripts in the simple low maintenance natural language scripts that can be executed by the AI Automation platform and virtual workers.
Yes, Pinnacle can use Enginuity’s AI and Machine Learning capability to help with conversion of most common traditional automation scripts. This requires the support of Pinnacle’s technical teams to provide the conversion to our clients.
Yes, QMFactory allows for the creation and execution of manual tests. However, because Enginuity uses natural language test scripts these can be easily followed and executed manually if required. Effectively this is what the AI automation platform is doing with virtual testers.
Yes, but we suggest this is not a good use of your time. To physically watch automated tests execute we recommend only doing this when you are debugging a test when its being created. When tests are ready and packaged up to be scheduled for execution by the virtual testers, these can not be observed being executed.
Yes, Pinnacles products can be integrated with a number of common code coverage tools. It is often desirable to instrument the code and use Enginuity and the code coverage to validate and ensure all code coverage path options have been tested.
The platform scalability in AWS and Azure is infinitely scalable and will be spec’d and sized to meet your teams demands. The same is true for the virtual worker execution platform, this can be sized using either 1 or 1 million virtual workers. The limitation is typically in the sizing/ performance of the test environment which may struggle with the volume and throughput of the tests being executed.
No, Pinnacle has designed and developed its products to embrace innovation and not technical complexity. This, therefore, means that our natural language test scripts can be created by all team members and does not require the technical coding skills typically used within traditional automation tools.
Yes, the AI automation scripts are simple and easy to read, understand, create and maintain. This, therefore, lends itself to all testers and business user teams being able to work collaboratively to create and run automation.
Yes, Pinnacle’s Authorised AI automation platform was created specifically for this purpose. Authorise takes business user acceptance criteria and automatically executes the processes, tasks, transactions and reports required to satisfy the users acceptance criteria. Upon successful completion of each item the activity is rerun and a run time video is captured and posted to the users online dashboard for review and approval. Automating user acceptance activities significantly reduces time wasted by business users conducting multiple iterations of testing and defect retesting, allows them to focus upon their day jobs, and reduces to user effort to short familiarisation sessions.
Yes, you are able view and report on data from across projects as long as your user access has been setup to support this. Typically, this is capability is standard to Test Manager, Project Manager, or a Program Manager user access rights. With this level of access to multiple projects you can generate your reporting and real time dashboards views from across multiple projects.
Pinnacle’s AI automation platform is able to spin up large numbers of virtual testers, and they can also capture the user response time whist conducting activities, tasks and processes. Typically, our customers are using the products for baselining system performance and capturing user response times as part of an ongoing regression pack for field validation exercise. It is also possible to capture server, network, and latency data via separate integrated tools. This therefore enables you to improve the ability to conduct root cause analysis as to why you may be seeing performance issues.
Yes, Pinnacles Authorise product can support compliance, audit and regulatory validation. Within the product you can build a custom guidebook for what is to be validated and verified. Where an existing compliance or regulatory guidebook exists then it is quicker to use this to assess against.
Yes, Pinnacles web or SaaS platforms can support this requirement. Please remember that where sensitive data is used, this needs to be held and consumed within the country of origin. Therefore, for Australian customers this requires the product databases holding the test data and scripts to be located within Australia.
Yes, Pinnacle’s open API’s allow for integration to many common vendor support systems, and can be used to link raised tickets with tests for execution and defect fixing.
Yes, Pinnacle’s products and AI automation is used to quickly test and validate support changes to production, along with ensuring that suitable regression packs are run prior to automated or manual deployments to production.
All customers that have an active licence for Pinnacle’s products all have a defined support and maintenance contract. Pinnacle and its team are always online and available to support service request items on the products and their use.
All Pinnacle products have formal training and certification available. Pinnacle can provide informal or structured online and classroom-based training. For customer teams where they have multiple Pinnacle products from the Quality suite, product specialist certification is available. Where customers require specific technical support for machine learning or customisation this is available from our client service catalogue for existing customers with an active licence, along with a support and maintenance contract.
The very vision and purpose of Pinnacle is to use leading technology and innovation to simply Agile, DevOps and Automation. To enable, non-technical and business users to be able to conduct automation of their systems and projects without the need for technical coding skills and maintenance overheads.
The very fact that we use no code and natural language scripts means that all team members can easily read and understand tests and the coverage that the team has.
Pinnacle’s technical teams can support the transition of service and asset to ensure an effective handover of assets at the end of service. This may include activities such as database copies, extracts, or automated conversion of test scripts from Pinnacle over to another open source or vendor product.
QMFactory is used to support the test management and test delivery process all in one product. Your test teams can design their test coverage, build and store their test assets and use the reporting and real time dashboards to track report on progress. All Pinnacle’s AI Automation products are fully integrated into this centralised management platform so that team are able to improve their collaboration and asset reuse.
QMFactory is used to support the test management and test delivery process all in one product. Your test teams can design their test coverage, build and store their test assets and use the reporting and real time dashboards to track report on progress. All Pinnacle’s AI Automation products are fully integrated into this centralised management platform so that team are able to improve their collaboration and asset reuse.
Yes, Pinnacle’s technical teams can help clients migrate content and databases over to QMFactory so that previously created assets and history are not lost.
There is a few ways that Pinnacle’s products support integration with an automated CI/CD pipeline process. It depends on what is required and what pipeline automation solutions are being used. Integrating QMFactory to the CI/CD pipeline so that tests are run, and results are collated within the centralised QMFactory product. Or may require a simple integration of the CI/CD process to Enginuity to have it scale its virtual testers and run the tests required. |
Pinnacle’s technical teams can help clients to define the best option that suits your organisations requirements, this may include:
QMFactory is available as a local on-premise client install, or via the web, or via a SaaS platform. Pinnacle can provide a hosted service or can configured to run from a client’s specific cloud or data centre.
QMFactory offer a wide range of standard reports which offer out of the box reporting capability for a large range of areas within the test process. In addition, custom reports can be requested and provided by Pinnacle technical teams. The same applies to all the real time dashboards that are available for users.
Not in the current version as this requires SQL access and custom queries on the database. Customers request custom reports via their support and maintenance agreement. The next version of QMFactory (due early 2022) will include the ability to configure reports.
Yes, QMFactory dashboards can be customised for specific user roles and can be applied across projects. The next version of QMFactory (due early 2022) will have extended dashboard capability.
For specific roles such as Test Manager, Project or Program Managers, these roles will have the need and the ability to see data and be able to report across several projects. For testers or developers or business users for example they are working on a specific project and can only see that information. These users can be allocated to multiple projects, but their work and reporting views are project specific.
Yes, Pinnacle licencing, installation and technical teams can support the installation and setup of the products to reside within your local data centres and test environments.
This is possible; however, this requires support from Pinnacle as the process will require either direct data base access or access to extracted QMFactory data sets.
Yes, manual tests, and exploratory tests etc can be created, stored and results captured for in QMFactory. Manual tests can also be scheduled and allocated to individual team members to ensure an even workload or technology, or functionality specific coverage is tested by designated team members.
Also known as no code test automation, scriptless test automation is a revolutionary testing model where testers can create fully automated testing without any code at all. Traditional automated testing requires complex script development and requires strong engineering skills. Low code requires minimal amounts of coding, and scriptless or no-code automation specifically means no code is required at all.
No. Enginuity is a no code test automation platform for software testing that requires zero coding to create, run or maintain the automation scripts. Automation in Enginuity lets non-technical users get more involved with automation in projects, so that project transparency and collaboration is increased and duplication of effort between manual and automation tests are removed. Enginuity accelerates delivery of software and overall project deliver, and reduces the need for siloed testing, development and business/ non-technical testers within project delivery processes, and a more inclusive team engagement.
No not with modern automation platforms. Traditional automation test tools required extensive coding and specialised skillsets, low code or no code automation platforms lets user’s setup, run, and analyse automated tests without the need for code.
Enginuity tests are decoupled from the application under test, and not requiring code for test automation scripting means lower effort and maintenance when applications change.
Change happens, and often. Enginuity has an AI scan process that can be scheduled to run, which allows Enginuity to find where changes have been made to the application that the AI has not yet learnt or will break existing script coverage. With a quick AI learn activity for the applied changes this will often resolve this issue. Where new functionality or features are added an update to a reusable script component ensures that all scripts are now ready to support the given change.
Yes, for Enginuity to recognise the system under test and for it to be able to correctly execute the natural language test scripts and actions, Machine Learning is conducted. This machine learning is a technical process that is conducted by Pinnacle technical teams and once complete allows for tests to be created against the UI, API’s, message/ file layer, services, interfaces and comm’s or the system databases.
Enginuity can provide a comprehensive set of standard and custom reporting on execution (pass/fail) results, error information, virtual user status and batch processing.
Results are stored within the central repository of QMFactory so that historical data is available. If Enginuity is being used as a standalone product results storage is subject to file size limits that are set within the environment configuration.
Yes and No. Yes, you can run virtual workers whenever you want 24/365. However, in most instances your virtual workers will be controlled by your licence agreement and that means individual projects are allocated a specific number of virtual workers. If there is a company wide agreement that all virtual workers are to be pooled and work executed upon a first come first run basis then that again yes but subject to allocation of virtual worker loads.
Yes, Enginuity can be used to automate the testing of an application UI and can run end to end business process tests across a number of applications as a user work use them.
API (Application Programming Interface) Testing reviews the integrity of the API’s logic at the business layer. API is tested for its ability to accept and return information from various user interfaces (UIs) and numerical transactions on the database layer. The test also reveals if the API meets functionality, reliability, performance, and security requirements.
API testing can be automated for faster feedback, reducing time for debugging and patches. Backend APIs are used by several clients and servers. Automating your API testing is essential to ensure all software functionality is working throughout development.
API testing is done at the business layer as part of integration testing to ensure that all connected systems can exchange data seamlessly. API tests reveal a systems functionality, reliability, performance, and security. Because a functional API is integral to the system’s success, developers will perform API regression tests during application development and every version update to verify that all changes added will not break the API interface.
There are a number of test automation tools for API automation testing. However, Enginuity uses its same modular, natural language (no code) script format to test the entire end-to-end workflow with integrated API and UI tests through its single platform. Enginuity automated API tests can validate status code responses, headers and body without having to interact with the application’s UI. Changes made to the API are easily adopted using our change only the impacted module philosophy, for low effort and maintenance automated API testing.
Yes, Enginuity is available upon Pinnacle’s AWS and Azure platforms. We can also commercially agree to organise in certain circumstances that Enginuity and Pinnacle’s suite of tools can be made available upon a clients own Cloud platform where required.
Yes, Enginuity and Authorise with automate the execution of tests and acceptance validation (processes, tasks and report generation) across many of the common bowsers, supported OS builds and on virtual and physical devices.
Yes, Enginuity virtual testers can start and end timers and capture end user response time for running transactions within the application under test. Timings are captured within the UI to ensure that the real user device timings, search timings and screen rendering times are known.
Yes, Enginuity can execute tests on both virtual SDK’s and physical devices, and can be integrated to execute tests across a number of devices, build and OS configurations that are available within many of the commonly available cloud enable physical device labs.
Yes, if your user profile has the permissions to schedule and execute virtual tester batch runs, then yes you can define what tests can be run anytime 24/365. In addition, it is also possible to set up the automated batch processing capability which allows you to define the tests that require to be run at a specific time allowing for scheduled unmanned execution for overnight runs.
Inside the virtual work scheduling panel, you will see that the status of the virtual workers. When a virtual worker or a number of virtual workers are running a batch of activities, their status is updated to show they are working. Pinnacle does not believe that it is a good use of people’s time to watch the tests run. As any failures are reported made available for action within the reporting dashboard. It is not possible to watch virtual testers execute their workload, this is only possible when watching a script be executed within debug mode and not within the batch.
Yes. Enginuity works with legacy technologies such as mainframe, green screen, etc.
Yes. Enginuity will work a variety of SQL and non-SQL databases and multiple integration platforms. Since Enginuity tests are technically decoupled from the architecture, it is possible to execute an Enginuity test case across web, mobile, client-server, mainframe, integration platform, and DB within the same end-to-end Test.
Enginuity Tests utilise advanced AI scanning processes to detect application changes allowing a certain element of self-healing of the Test without the need to maintain the Test. Reusable components mean that a single change in one component applies to the change to every Test using that component, which significantly reduces the maintenance effort. And, since the Enginuity Test is not coded to the application, it is not necessary to make changes to the Test when the application changes. The maintenance effort for coded scripts such as Selenium is significantly higher because of the widespread changes needing to be made to the Selenium script.
In this unlikely event, it is possible to extract the Enginuity Test into different formats. For web applications it is possible to convert Enginuity scripts to an equivalent script for a different toolset.
Yes. Screenshots can either be included within a Test to capture a specific view or specified as a parameter so that all Test capture screenshots during execution. Enginuity captures a screenshot automatically when any warning or errors are encountered.
Yes. This is one of the execution parameters. You have the option to create defects automatically during execution run time and Enginuity will populate all of the details onto the Defect for you. Enginuity will automatically detect if the same error is repeating and will only create 1 Defect for multiple failures. Each Defect is created with reference to the Tests/Steps where this failure occurred.
Enginuity Tests are written in structured English and describe the activities to be carried out (e.g. enter username, enter password, click login button) so there is no need for any coding skills. Enginuity helps you create the Test
Since Enginuity is not coupled to the application, it is possible to write an Enginuity Test directly from the User Story or Feature File; the application is not needed. This means that Enginuity can execute Tests as soon as the application has been deployed into the test environment and the AI scan has completed.
