(03:28) it's usually, I don't know, 99% of cases that I saw is sometimes the infrastructure of the client, and you need to convince them to check there, and then they will be able to fix the problem that they're facing. Yeah, it's not just like sending them a simple FAQ. It's not possible in our area, unfortunately, or maybe fortunately, I don't know. Well, sounds like they got the right guy for the job. Um, so I guess how does Allure TestOps facilitate the centralization of both manual and automated test cases? What impact

(03:58) does this have for overall QA efficiency? But efficiency, okay. Uh, the team which stands behind the Lotus stops, uh, they came from the automation, and they built an automation-first test management system. So, basically, they were thinking about all the shortcomings that they saw in the TMSs back then. It was maybe started somewhere in 2018 or 17, the initial version of Allure Enterprise Edition and tests now, and then they worked with the testing teams that were responsible for the manual testing as well, and then it was combined all

(04:34) together. So, you don't need to have different tools. You have one project. In one project, you have both automation tests and manual tests, and all the statistics, all the data that you're producing when you're testing your system actually stays in one place. So, it's much better for the teams to have it in one place. They still can distinguish between the manual tests and automation tests because you filter out something, and you just see what you are working on, and it's much easier to have this tool rather than to go to different

(05:03) tools and then collect this information from different places. And also, we take the heavy lifting of the integration with all the systems. So, for example, with the CI system, with the issue trackers, you don't need to do that because it's already done. And if you have something in your mind that you want to integrate the tool with, then it's possible for us to do it on our side. So, you don't need to think about the integrations. You just think about stuff that you need to test. So, basically, this is your product, and you're testing your

(05:29) product, and that's it. Yeah, so you guys, uh, one of your core competencies is really allowing people to visualize the data, right, and identify bottlenecks. Uh, could you talk us through maybe some more of the uh, data analysis that your platform does to help people identify issues with their testing? Ah, yeah, uh, we need to start with the Allure framework again because the Allure framework was built based on the idea that you need to provide some additional data to your tests, otherwise, people won't be able to understand. So, there are two

(05:58) groups of people. The first group writes tests, and sometimes they are not the people that are analyzing their test results. Yeah, so you can add additional information that we reference as metadata, and metadata allows you to see uh, the test results and generally information about your product in different slices. So, it could be a slice for product analysts, it could be a slice for, uh, I don't know, developers. Developers are interested in what is my module, my microservice doing, and the analysts are interested in the

(06:29) feature, so they need information about the feature slice, and the same data could be used for different people. And you can use this information just to narrow down searches, and based on the information that you provide from your test, from your test results, you can provide analytics for different types of people. So, if you want to see microservices, then you just filter out everything that is related to your microservice you're working on. If you want to think about the feature, let's say call it authentication, that you will be able to

(06:58) see only how your particular feature is doing and what are the stories. And also, not only analytics but the generally representation of your test cases, which is documentation of your test, can be different depending on who you are and what your need is. So, it's possible to move this data like this and provide and have different views from the data that you receive. Do you have any best practices for maybe customizing the dashboard for, say, business stakeholders that might have nothing to do with the technical side

(07:29) but just like to know kind of how the product is? Yeah, yeah, yeah, yeah, that's correct. So, if you are just a manager, then you just ask your people, "Please give me this slice of data. I just want to see global numbers," and it's also possible. So, you don't create any filters. You just show general information. If the dashboards that we have in Allure TestOps is not enough, you can gather this data from the database if you installed Allure TestOps on-premise, and it's also possible to gather this data and build some Grafana dashboards which

(08:02) your managers would just love. Nice. And what kind of data do you think business stakeholders would be interested in, like performance data, maybe like improvement based on the testing results? Uh, usually people are looking for how the automation is progressing because you have some manual tests, and manual tests take usually a lot of time, and you need to at least automate your regression testing, which you run over and over again day by day. So, if you have something on the regress side, then you probably want to automate it,

(08:32) and if you want to see what is the progress of basically the number of automated tests today, the number of manual tests today, then tomorrow you have this, and the number of manual tests decreasing. So, basically, you can see this progress, and you can understand what will be the time consumption for the regression testing in the future, for example, like that. So, this is what they're interested in, and they are interested in, how to say that, decreasing the cost of manual work. Yep. And can you ever get to a full point of

(09:01) automation, or do you still recommend some manual tests just for a sanity check? It all depends. It depends on the team, depends on the product. Some products do not have automation at all, and some products do not have manual tests at all. It all depends on the team's decision and the approach they selected. So, there is no right way and silver bullet. Yep. So, what considerations would a QA team have to think about before moving over to Allure TestOps? How do you guys support this transition for teams

(09:31) that might have been doing this manually, the old-school method, moving over to your platform? What kind of support do you offer those type of companies? Okay, well, the beauty of the Allure framework again, uh, is that you don't need to create manual, not the manual, the automated test cases, because when you upload the data, automated test cases have been created automatically by the tool. And then we also have scripts for the migration from third-party TMSs. So, basically, if you want to migrate from software, we can create or update

(09:59) our existing scripts for API-to-API migration, and then you just migrate, check, and continue working as supposed before in a different tool. Nice. And if you need some advice on how to use the workflows inside TestOps, then starting from the first day of the trial, uh, you also can ask us to help you assist in order to understand what is the better way to handle this. That's nice. And uh, that's kind of your job, right, handling customers? Yes, what I'm doing. So, I'm not lying at all. Yeah, you know the process through and through. Um, how

(10:32) long does that migration typically take for companies? Uh, depends on the number of tests on the source system. Uh, you need first to create some, let's say, first attempt in order to understand whether we can process everything that is on your side of the third-party TMS. Uh, then you can create a production migration and just start working. It could be one day, could be two days, depending on the number of the projects and how fast you can check uh, the test cases that you created with the migration tool on the Allure TestOps

(11:03) side. It's really not too bad in the grand scheme of things. Nice. Um, so I guess, could you give any examples of, you know, some companies that you've worked with where, you know, their QA teams moved over to working with Allure TestOps and what the results of those transitions were? Okay, off the top of my head, it's not a usual example, but I think it's a good example. Sometimes you have a product, and the product is not for you, not for yourself, but it's a product for some company that you work for. Yeah,

(11:36) let's say you are a subcontractor, and there is a very good and big team that is working on it, and you need to deliver the product, and the product delivery depends on whether there are some critical errors or not. So, a couple of teams, they created so-called quality gates based on Allure TestOps because Allure TestOps allows you to see the data in real-time. So, you can see whether your tests are producing errors. So, basically, maybe something is wrong on the test side, or maybe something is wrong on the product side, and you are

(12:07) able to see and gather this data and make a decision. Maybe in an external tool, Allure TestOps doesn't have this capability at the moment, but we are thinking about it, and you can gather this information. You can see 100 errors, and two of them are critical, so you just stop your pipelines and stop delivery, rework, and then run it again. So, they were able to save this time on incorrect product delivery on the side of their client, for example, like this. And also, we are not only limited by the managing tests inside the system, also they have, let's say,

(12:43) an ecosystem around it. So, there are a couple of plugins, a couple of plugins for CI systems, and a tool that allows you to upload the data. So, you don't need to have this overhead in order to integrate with TMS. Basically, you just use our tool. It uploads the data. If you want to automate something, then ID has the plugin, and the plugin allows you to gather this data from Allure TestOps and create a kind of skeleton for your automated test and then automate it. You just need to add the code, the actual code on top of the steps that we are able to use.

(13:17) So, it saves you time. It allows you to save time, and it also creates a bridge between the manual testing and automated testing, so it also saves time and eases the process. Great. And what else do you guys do to ensure scalability and adaptability for, you know, going forward? So, once the QA team adopts Allure TestOps, how do you guys ensure their future success and optimization of their use of the platform? Uh, when you are using Allure TestOps, it's, let's say, it was designed as a cloud-native application, so it runs

(13:50) easily in Kubernetes. We have our SaaS system. It runs on the Kubernetes clusters, and if people are deploying it in-house on their premises, then also if your workload work becomes bigger, uh, it's inevitable, I think, and then you just need to scale it horizontally. You don't need to add anything. You just add additional resources, and on top of that, you will be working as it was before. So, basically, you don't need to rebuild the system. You just need to add additional resources for that, and that's it. And we also can, how to say that, provide

(14:27) some advice in order to scale up the system, tweak some parameters in order to enhance the performance of the tool. That's great. How does your guys' pricing model work, uh, you know, for say, an entry-level team coming on board, and you know, maybe somebody that wants to get to a higher level of scalability? Uh, when you're working with the SaaS system, that will be, I think that's 39 bucks per one seat. One seat that means one person is working in the UI. When you need to add some additional persons, additional teams, then you just add seats. If you have

(15:01) some managers that require just access to dashboards, and they don't need to create anything, that will be free of charge. For the on-premises solution, it will be the same, that price will be maybe a little bit different, and you just need to add additional licenses if your team grows, and that's it. Yeah, absolutely. When your code base grows, and the number of tests is growing, then you just need to add resources, not the seats, not the license, it doesn't change, depends on the workload. Yeah, I can't really put a price

(15:34) on testing. I know it's like some of the more boring work that you could ever do on a product or a platform, but it's, you know, one of the most important things you could obviously do. Yeah, uh, testing is not boring actually. If you are like a monkey, what does this button do? It's a lot of fun, to be honest. Yeah, to break things. Yeah, yeah, break things. Some people love it, some people do not love it, but then they are not in QA. Yep, yeah, true. Yeah, I could see that too. I guess I'm always breaking things in marketing, too, you know, we're

(16:06) calling a candidate for testing. Yeah. Yep. Yeah, I've tested plenty of apps over the years, you know, I've been an entrepreneur myself that brought many products to market, and yeah, I guess you're right. Yeah, testing is pretty fun. Fun. I love it. I love to break things.

Yep, yeah, I've, I've done web apps, mobile apps, I've done IoT products too, so like, yeah, the IoT products are kind of fun when you break those. Um, anyway, so I guess, uh, failure comes with any profession. Can you tell us about a time you failed and what you

(16:35) learned from it? Yeah, uh, so we're going back again. When the tool was built, it was built with automation first in mind. So, basically, it was built by automation QAs for automation QAs because all the people standing behind the system were for, in general, automated test. One was from, how was, what is the name of the profession there? Something like a developer in test, let's say. And they were building the tool for themselves initially. And yes, so when our client base grew, it was clear for us that probably manual

(17:17) QAs felt a little bit uncomfortable with the tool because it was for automation, and some features that manual QAs were used to seeing in tools like, for example, TestRail, were missing. We communicated with the teams a lot, and the whole year of 2024, we dedicated ourselves to closing this gap. Mhm. So, they got, for example, shared steps, which allows you to reuse some steps that you described in your test many, many times. So, if you, for example, are testing authentication before the main test body, then you can use this shared step and

(17:56) just put it over the test body of your test, and it will be in your test scenario, so you don't need to write it again and again. Then again, people like to see, for example, some formatting in the text, and it was completely missing. We just had plain text, and plain text wasn't comfortable for people because sometimes you just lose something if it's not highlighted or it's not bold or whatsoever. And also, we used, how to say, the substep approach, and some people just didn't get it, and we thought about it. Now, the test

(18:30) cases for many have a dedicated field that codes the expected result. It was possible to use the expected results before, but now it's explicit. They love it, and they use it. So, the initial, maybe, mistake was to think only about one group of people that is working. So, we say that we are for manual QA and automation QA, but then one particular group that was heavily dedicating themselves to working with the tool was just lost a little bit, and they felt uncomfortable. So, this was our mistake, and what we fixed last year. Yeah, I feel like that happens to a

(19:08) lot of entrepreneurs and a lot of people building products. A lot of times they build a feature because they think, you know, this is interesting for me, but not necessarily for their customer base, and they'll go and build it and then realize that it doesn't really connect with the audience. Yeah, that's correct. So, they call it confirmation bias a lot. Yep, yeah, we get that all the time. Yeah, we get excited about building features, and you know, but if you work backward and you understand the customer and you know really what

(19:34) their problem is, you could, uh, you know, really align your solution a lot better with that. Yeah, sure. Awesome. So, I guess, I guess what kind of books would you recommend, I guess, about your field that somebody could pick up and learn more about what you do and how they can become better at TestOps? Uh, about my field, if talking about, uh, technical support or about, you're talking about the testing? Actually, more about the testing. It will serve both, uh, the people for working with the technical support and testing. Uh, last

(20:03) year I read, not read, but reread books by Nigel Poulton, "Deep Dive" and "Kubernetes Book," which are written really, really good. I like it, and sometimes I just need to refresh my memory. If I don't do something in the infrastructure, then you inevitably forget it, and I need to refresh my memories. And maybe a book by Lorie Gonzalez, which is called "Deep Survival." It is not a technical book, but it's interesting stories about people who died and why, and who survived and why. So, basically, it tells you if you're an expert in your field and you are

(20:42) overconfident, then the probability of your dying in the field is higher, much higher. Yeah, so you need sometimes to go back and think like you are just a newbie and see things from different angles because, uh, what I saw in my work, uh, sometimes overconfident people just can create a disaster. 100%. Yeah, so probably these three books I would recommend to read people in tech, definitely in the technical field and generally, especially in QA. Yeah, I like that you'll be overconfident. Oh, totally. Yeah, ego kills everything, right? So, I think, um, you know,

(21:24) people's biggest weak points are either they're insecure or they're egotistical and they think they know it all. And, you know, as the saying goes, like, a true wise man knows that he knows nothing. So, you need to, you know, always kind of assume that you know nothing, like you said, and kind of, yeah, yeah, take a look at everything from a rudimentary level and just assume that you really don't know everything that you know so you don't introduce that confirmation bias. And, you know, I've seen that with entrepreneurs many times.

(21:47) They'll bring a product to market. They'll say, "Everybody should understand what I'm doing here. They should love it for what it is." And it's like, well, did you actually work the other way around and, you know, actually understand the customer's problem and then try to put yourself in their shoes and design an experience around it that would deliver an elegant solution for them and actually help solve the problem? Yeah, exactly. Great. And I guess, um, any other philosophical takeaways that you've learned over the years in your field, and

(22:14) uh, you know, maybe some rules of thumb that other people could follow in your footsteps and learn from? Interesting, let me think a bit. In my case, and I'm talking from the position of the lead of the support team, uh, you need to be curious, otherwise you cannot learn. And in your field, it's always, how to say that, even if you don't use some knowledge right now, it could be used tomorrow. Otherwise, if people are not curious, they are not learning, and they're not progressing at all. So, you need to be curious. Yeah, that's a great point. Like, uh, if you stop

(23:03) being curious, you kind of start to die in a way, right? You know, uh, the moment you stop having that desire to actually learn and improve yourself, you kind of, you kind of wither away, and, you know, professionally, intellectually, and, you know, the likes of that. Yeah, absolutely correct. Yeah, I've always seen that too. The people that are always curious, even if they fail too, they're always curious about, you know, how to fix that problem or, you know, maybe even the next project they, uh, they move to, you know, they're, they're curious to, like, you

(23:30) know, they have that passion to really figure it out. Um, there's a saying I always tell people too: the person that loves the journey will always go further than the person that likes the destination. So, I think that the love of that journey is that curiosity that really drives people. It drives people into it. It allows you to find unconventional ways from the situation that you are currently in. Yep, yep, definitely. Um, so I guess, is there anything else you'd like potential customers to know about Allure TestOps,

(23:55) and why they should pop over to your website today and pick it up? If they go to our website, they will actually be able to request a trial, and the trial will allow you to understand whether the tool fits your needs or not. You just need to be curious again. You just need to be a monkey, and then you can try. You can see whether it fits in your infrastructure, it fits in your processes, and you just can use it or not. Or at least you can try Allure Framework, which is a really good tool for automation. Uh, if you want to see what are the test results of

(24:27) your automation tests, and you, for example, for some small teams, you just don't need a big TMS system, but you need just information about what is happening right now, and the tool can allow you to, uh, allows you to see what is happening. Great. I guess what's next? Every tool has its own user, and initially, if you don't need Allure TestOps and you are satisfied with Allure Report, uh, but in the future, maybe the history will change, we'll see. Yep. And I guess what's next for Allure TestOps? You guys have any exciting new features or, uh,

(25:04) strategic directions you guys are taking the company in? Uh, I can say some buzzwords like AI or something. Yes, for sure, it will be definitely incorporated in the tool, but if we're talking about the product itself and how we are going to evolve, probably in the future, our SaaS product will be easier to use for the end-users, uh, easier to use generally, how to say that, for the onboarding purposes, it will be easier, and we are going to evolve and also incorporate some additional features that are currently not in the tool, like, for

(25:37) example, managing the requirements. Some big teams are requiring this, and they want to see the requirements for the product inside one tool, and then link your test to requirements, link your task to requirements, and we also heard this user voice, and we are going to implement these tools. Excellent. Um, so besides being able to pick up Allure TestOps and NachoNacho, the SaaS marketplace, the best way to buy SaaS, where else could people learn about your guys's product and, and, uh, could you say the website name

(26:10) one more time? Uh, it's qameta.io. Yep. Uh, people go there. People sometimes looking for Allure Report without knowing that they probably need TestOps because they heard something, and they also saw "jungle radio," how you call it, uh, when people tell each other about the tools they're using at their workplace. Sometimes people change companies, and they say, "Why are you using this one? Look at this!" And this changes the path of our tool. So, sometimes people don't even know about it, and still someone comes from another team

(26:46) and then they switch their team's existing tools to Allure TestOps. So, it's both, and just some searching on the internet. Great. Well, George, I really appreciate you coming on today. We're very excited about this partnership. I love having more TestOps products on our platform. You know, frankly, I think we need more of them, so glad to have you guys. Um, you can learn more at qameta.io.

(27:11) Um, you could also pick it up today in the NachoNacho marketplace. We are the best place to buy SaaS. We have a great deal on this product. So, feel free to check it out on the website to learn more or pick it up today in the NachoNacho marketplace. Thanks for having me. See you. Thank you, George. Bye.