Product Optimize Breakout Summit/2020: The Future is Now! Using Machine Learning to Automate Plugin Updates + Secure Your Site
Did you know plugin vulnerabilities represent 60% of the known entry points for website attacks? Learn how Smart Plugin Manager uses machine learning & visual regression testing to automate plugin updates and keep your site secure. Bonus: get a preview of exciting platform enhancements launching in the next coming months.
Annan Patel, Senior Product Manager, WP Engine discusses:
- How to use machine learning to manage plugins and keep them up to date with Smart Plugin Manager for automatic updates.
Why visual regression testing? Things like a broken call to action or a missing form field – that’s a broken site. That means your user can’t use your site the way you’ve intended.
Full text transcript
[00:00:01.04]
– Welcome everyone to WP Engine Summit 2020, the digital breakthrough conference. I’m excited to talk to you today about our topic, the future is now, using machine learning to automate plugin updates and secure your sites.
[00:00:13.08]
My name is Annan Patel, just to introduce myself. I’m a senior manager on the product management team here.
[00:00:19.03]
Let’s talk about managing your plugins. I always love these slides that just show a few numbers, and here’s my slide with some numbers.
[00:00:24.09]
On average, a WP Engine site has 16 plugins. Our own security team here at WP Engine has identified 144 different plugin vulnerabilities just in the last six months.
[00:00:37.00]
Finally, through surveying and talking to our customers, we estimate on average, a customer spends about eight minutes updating their plugins manually. So, you’d imagine, if you have 16 different plugins that have, different updates throughout the course of a week or a month, and you’re managing multiple sites, this is going to add up to a lot of time, but we know that this is important because keeping these plugins up to date is the easiest way to keep your site secure.
[00:01:02.07]
So, by not updating your plugins, you’re introducing unnecessary risk. We also talked to our customers about managing their plugins and two of our largest customers gave us some pretty interesting insights.
[00:01:14.00]
Marriott has a person who updates plugins across their entire portfolio of sites, and they know that that’s not the best use of their time.
[00:01:22.09]
Thomson Reuters, they know how valuable the WordPress ecosystem is, but they also know that it’s third party software and third party software introduces potential risk.
[00:01:32.05]
And so, ensuring that you can be up to date on your plugins, again, is the easiest way to de-risk your site.
[00:01:40.00]
So, let’s talk about our solution. We launched Smart Plugin Manager last year, mid last year, and we believe it is the best plugin management tool on the market.
[00:01:50.03]
It’s the only plugin management tool that will automatically update your plugins and then deliver success criteria with visual regression testing.
[00:01:58.02]
And we’ll talk about that more here in a little bit. Smart Plugin Manager is also fully automated.
[00:02:02.09]
And so, if we think that your site has broken after a plugin update, we’ll roll it back and let you know that, “Hey, we’ve restored your site to its previous state so that you can take a look at those plugins to understand if those visual changes are acceptable or not.”
[00:02:17.05]
So why visual regression testing? We know that some visual changes can just be difficult to identify. You know, things like a broken call to action or a missing form field, that’s a broken site. That means your user can’t use your site the way you’ve intended.
[00:02:32.08]
And sometimes these changes are just small pixels and it’s just not reasonable for a person to be able to identify those changes, particularly across multiple pages on multiple websites.
[00:02:43.09]
This is a costly endeavor and it’s one that’s just begging to be automated.
[00:02:47.06]
So with our visual regression testing capability, we’re able to identify these changes at scale, and actually let you know that your site might have been broken by a plugin update.
[00:02:57.04]
So, in terms of how we use visual regression testing, when we know that your site has plugin updates available, we take a screenshot of your homepage and some other pages on your site, based on your site map, we will run the plugin updates, and then take another set of screenshots.
[00:03:14.05]
We’ve also taken a backup of your site during this process. We’ll then compare that before and after screenshot, do a pixel by pixel comparison to actually identify areas of change.
[00:03:24.06]
And then our machine learning model will actually classify those visual changes as either significant or insignificant, and you can take a look at these examples. cause we know that some visual changes are fine.
[00:03:34.08]
They’re expected, things like a captcha, of course that’s going to change. But when it’s a significant change, which is critical site functionality broken, or a significant change that will dramatically alter the user experience of your site, these updates are significant, these changes are significant, and we deemed these updates unacceptable.
[00:03:53.06]
In this case, we’ll roll the updates back to your previous state and let you know.
[00:03:57.04]
So again, you can check these changes out for yourself and understand what might be causing them.
[00:04:01.09]
So what’s new, we’ve made a lot of great investments in Smart Plugin Manager over the last several months, I’m excited to share a few of those updates.
[00:04:10.09]
So, improved visual verification. We’re constantly investing in our visual testing capabilities because being able to classify, being able to identify and classify visual changes on your site is really why Smart Plugin Manager is so essential to our customers.
[00:04:26.02]
And so, we’ve made big investments so that when we’re actually taking screenshots before and after updates, we’re able to identify the specific frame of an animation so we can better classify
if a change occurred, and if that change is significant or insignificant.
[00:04:40.02]
This has also led to a faster plugin updates process.
[00:04:44.04]
We know that keeping your site up and available for your customers is top of mind and the quicker we can conduct updates and the quicker we can verify that those updates were successful or unsuccessful, the quicker your site can be back up and your users can get the value out of your business.
[00:04:59.02]
We’ve revamped our user experience. We’ve made big investments in improving our onboarding
and bulk site management capabilities.
[00:05:08.04]
Today, we believe Smart Plugin Manager can truly be a set it and forget it type of tool, and we wanted to create an experience that really made that easy.
[00:05:17.07]
So, when you get started with Smart Plugin Manager now, we’ll actually walk you through how to enable your sites, configure the settings to your needs, and really just set it and forget it.
[00:05:28.03]
And you can do this for one, dozens, or hundreds of sites, all in one seamless experience.
[00:05:33.07]
And this is all available in WP Engine’s user portal, so no need to log into dozens or several WP admin instances, this can all be done within our portal.
[00:05:43.03]
And finally, we’re constantly delivering on what we view as customer inspired features.
[00:05:48.03]
Customer feedback, listening to our users, is vital to how we prioritize the work that we put into Smart Plugin Manager, and we’ve delivered on several features that we know address specific customer pain points that we’ve heard.
[00:06:01.06]
First, we now offer a weekly update frequency. If you’re a longtime customer, you know, we traditionally have conducted updates daily, as we believe that this is the best way to keep your site secure.
[00:06:12.02]
But we know that a lot of our customers have very defined maintenance windows for when they would like for updates to occur, to coordinate with other actions they might be doing to maintain their site.
[00:06:21.06]
So, by offering a weekly update frequency, we’re better adapting to some of those, new workflows and processes that many of our customers have.
[00:06:28.06]
Likewise, we now support custom plugins that are hosted in private Git repos.
[00:06:33.04]
We know a lot of our users and lot of our customers are building their own plugins and updating those along with the publicly available and commercial plugins that Smart Plugin Manager has traditionally updated is just as important to automating. So, we now support these plugins as well.
[00:06:49.08]
And finally, we know that visual testing is a great way to tell if a plugin update was successful or unsuccessful, but there are verifications that go beyond that.
[00:06:57.06]
And so now, one new feature we’ve added to Smart Plugin Manager is that we now check for title and tag changes that can impact things like SEO or the way that content is rendered on your social channels.
[00:07:09.06]
The user experience is more than just your website. It’s how your customer finds your site, how they interact with it across all of their channels, and then ultimately how they, they interact with the site itself.
[00:07:20.06]
So thinking about that entire process is now something that is top of mind for us as we invest in Smart Plugin Manager and SPM will now check for these changes and let you know, so you can get ahead of any impacts there might be to your search engine rankings or the way that content renders on channels like Facebook or Twitter.
[00:07:37.03]
So that’s what we’ve been working on.
[00:07:38.07]
Let’s talk about what’s coming. This is always the exciting stuff, right?
[00:07:41.06]
So, looking ahead, we really want to double down on a few areas that we’ve been investing in.
[00:07:46.07]
Let’s talk about usability and user experience first. We want to move more of that SPM experience into the user portal.
[00:07:53.00]
Today, if you’re a smart plug and manager customer, you’ve probably seen our emails where we let you know what happened, we’ll tell you what plugins are updated, and we’ll give you the actual screenshots if we’ve identified visual changes.
[00:08:03.04]
We want to let you actually compare those in an easier way. And we want to move more of this into our user portal.
[00:08:09.01]
So look for updates to the Smart Plugin Manager experience in portal that will actually show you the results of updates along with providing you that visual comparison capability.
[00:08:19.00]
Likewise, we want to provide historic update data so you can actually see the different trends across all of your sites and get insights about specific plugins is particularly to understand which plugins are consistently causing visual changes and which ones might be, you know, have no problems at all, which is always the goal.
[00:08:38.03]
And finally getting back to helping our customers, adapting to our customer’s workflows. We want to give you more visibility about when updates occur and what plugins will be updated.
[00:08:47.06]
And again, just to better accommodate your workflows and work within your processes.
[00:08:53.06]
We’re always investing in our visual regression testing capabilities. And this is another area where we’ll be making some significant investments for the rest of the year.
[00:09:01.03]
We want to continue to invest in our ability to identify site elements and get better at classifying them as significant or insignificant.
[00:09:08.08]
We want to make this process faster because again, the longer updates occur, the longer your site might be inaccessible to your customers, which is ultimately an impact to your bottom line.
[00:09:19.06]
And then, finally, as we’re adding more testing capabilities, we want to give you the power to choose which tests are the most relevant to you.
[00:09:27.02]
So as we add more, more granularity around our SCO checks, for example, or provide different
options for visual testing, giving you the power to choose which tests make the most sense is top of mind for us.
[00:09:40.00]
And finally, we know so many of our customers are agencies. You’re managing customer sites
and using SPM as a key tool as part of that process, which we thank you for, and we want to continue to help you serve your customers.
[00:09:53.03]
And so we plan on working on several features that we believe empower our agency partners.
[00:09:58.05]
First and foremost is client reporting. And this is something we’ve heard from a lot of our agency partners.
[00:10:03.08]
We want to give you the ability to provide an artifact, a document to your customers, to let them know the great work that you and WP Engine and Smart Plugin Manager have been doing on behalf of them to maintain their site, to keep it secure and up to date.
[00:10:19.03]
Likewise, we wanna provide even more granular update windows. We have the weekly update frequency now, but we want to get to a state where we can actually listen to our customers and give you, whether it’s by the hour on a specific date but that level of granularity, so that there’s more predictability.
[00:10:36.01]
And again, so we can better adapt to your workflows and processes and work around your developer’s schedule and not vice versa.
[00:10:43.00]
And finally, plugin updates is one aspect of plugin management, but we know that there’s so much more than that.
[00:10:49.01]
So we want to make the user experience a much more comprehensive multisite plugin management experience, giving you the ability to actually look at plugins across all of your sites, make better decisions about which plugins make the most sense based on your use cases, and then providing the key analytics and feedback that Smart Plugin Manager has gleaned over the past several months, that again, can help you make better decisions about the right plugin for the right job you have.
[00:11:15.06]
And, I guess, the final announcement I get to make, I get to do my best Oprah impression. We are now including one Smart Plugin Manager license with all of our premium
and enterprise plans on WP Engine for life.
[00:11:28.02]
So, if you’re a WP Engine customer with a premium enterprise plan today, you now have access to Smart Plugin Manager included with your plan for one site.
[00:11:37.00]
If you’re thinking about joining WP Engine and you end up joining with one of our premium enterprise plans, you’ll also get access to this.
[00:11:44.04]
So, this is a full unrestricted license that is yours in perpetuity for the life of your account.
[00:11:51.02]
We hope you enable one of your sites and can see the benefits of automation and using machine learning to secure and keep your site up to date.
[00:12:00.02]
So with that, I really appreciate your time. We’re going to stick around and we will be available to answer any questions you might have about Smart Plugin Manager, managing plugins, how we conduct visual testing, or really anything related to that.
[00:12:13.08]
So with that, we will talk to you soon, and thank you again.
[00:12:20.05]
– Hey everybody, it’s Monica Kervana, here from my home in Austin, Texas, happy to moderate questions with Peter Mochkow.
[00:12:30.06]
And, if I was talking to you live at the Austin Summit, I’d be asking you how I just did on my Polish accent.
[00:12:38.09]
Peter, I’m sure it wasn’t the greatest, but you guys, Peter is the original brains behind the original Smart Plugin Manager.
[00:12:49.03]
And so this is an exciting opportunity to get to talk to him based at his home in Poland, and talk about Smart Plugin Manager after Annan’s great session that you just saw.
[00:13:01.07]
So, we’re gonna look at the questions and have this amazing long distance conversation with a global audience. It’s pretty awesome.
[00:13:11.02]
So, Peter, first question for you, are there types of sites that the plugin updater, Smart Plugin Manager, does not work well?
[00:13:23.03]
– Yes, definitely. There are, in this kind of website, that we have problems with. These are mostly websites that have a big sliders and only random content.
[00:13:35.04]
So on every page load, everything is changing and there’s no constant, constant text or anything that we can see on every page.
[00:13:51.08]
– Okay, okay. Sort of an extended question from that, are there types of plugins that typically cause more issues than normal?
[00:14:05.05]
– Usually these are commercial plugins and you have to have a valid license subscription or software or something like that, to be able to download the update. And it’s not like those plugins are not working at all.
[00:14:21.03]
The problem is that usually people don’t have the valid license, they forgot to ring it up, and then SPM is not able to update such plugins.
[00:14:32.07]
– That is a good pointer. You gotta check those things to make sure. Okay, that’s helpful.
[00:14:40.04]
Here’s another one for you, Peter, does the testing detect non visual errors?
[00:14:46.08]
– Yes. So we are not only checking if there are any pixels change on your website, we test your website based on different criterias.
[00:14:59.09]
Like, a simple example, if we update a plugin on your website and there’s a PHP fatal error on your website will affect that.
[00:15:10.07]
And yeah, we’ll tell you about that, but that might wriggle in the change on your website, but there are also other things like there might be a PHP warning or notice that might be not visible on your website and your website seems to be working fine because there is no change.
[00:15:30.07]
But when you try to submit a form, it might occur that it’s not working because of that, that warning or notice, and will detect that and notify you about that.
[00:15:42.09]
– Awesome.
[00:15:44.07]
Here’s a question, I want SPM to do a pull request instead of updating the actual environment. Is this on the roadmap?
[00:15:55.07]
– No.
Right now we are only updating websites on an environment that is connected to, to the SPM.
[00:16:04.08]
And the reason behind that is, again commercial plugins, it will try to update your commercial plugins on a different stage and a different environment, it may occur that it won’t be working because the license is attached to a specific domain.
[00:16:24.02]
– Okay. I hope that answered your question.
[00:16:26.04]
Not able to see who’s answering what, sorry everybody out there.
[00:16:30.07]
Okay. Here’s another one for you, Peter. Can you share what client reporting will contain?
[00:16:38.03]
– That’s still being discussed,
[00:16:41.00]
but what I can imagine that the report will include, so definitely there will be information coming from SPM. So there will be plugins, which we have updated from which version to which when it has happened, whenever there was any problem with that.
[00:17:00.05]
And we are thinking about including some more stuff in those reports not only from SPM, there might be a wintering part and to let you know when your website was down and hopefully it wasn’t at all.
[00:17:17.08]
And, yeah. And there might be some other performance things that we might include in the report.
[00:17:25.05]
– Okay. Also, sort of in the agency client question category, can an agency use SPM and have it paid for by the client?
[00:17:40.08]
– Currently, we sell SPM as package and you have to buy a specific number of licenses.
[00:17:49.01]
And currently only you as an account owner have to pay for that. So you have to agree with your client how you want to charge him for that.
[00:18:06.03]
– Yup, okay. Got it.
[00:18:08.03]
When will one free license for SPM be enabled in the WP Engine dashboard?
[00:18:16.02]
– Okay, so you should already see those license available. Even right now.
[00:18:24.05]
– Yes, great.
[00:18:28.08]
Oh, here’s a good one, Peter. How does virtual regression testing work Annan mentioned machine learning, does this mean the underlying algorithm or model actually gets smarter after each transaction?
[00:18:43.09]
– No, so it doesn’t work that way. So we are in control of this machine learning model and it is a supervised learning.
[00:18:55.03]
So, to make it better, we have to teach our algorithm once again to improve.
[00:19:02.04]
And we are doing that from time to time when we gather enough data to try to improve the model.
[00:19:10.06]
– Okay.
[00:19:11.05]
Okay. This is coming as an additional question from the one that we had just previously, which plan type offer the free SPM license?
[00:19:24.04]
– So, all educators and clients should see SPM.
[00:19:30.06]
– All SPM Enterprise plans.
[00:19:33.03]
– Yes.
[00:19:37.07]
– Okay.
[00:19:39.00]
We’re bouncing around a little bit, Peter, sorry.
[00:19:40.08]
Just questions all over the map here. Is there a way to tune or guide the machine learning ourselves?
[00:19:52.09]
– There will be, not right now, but we are going to improve the experience
in the user portal where you can currently only change settings for SPM.
[00:20:06.00]
And we are going to provide more extended results of SPM. Cause right now you only get email notifications where there are some information that we have more details to share with the clients.
[00:20:21.09]
And we are going to share those data in the portal. We are also going to provide the feedback loop that the customer can give us a bit that you disagree with, with what our machine learning decided.
[00:20:43.00]
And this way you can help us to improve the model.
[00:20:48.02]
– Awesome. I love open source.
[00:20:54.06]
In the looking ahead section, Annan talked about custom reporting, can you share some more detail on that?
[00:21:03.09]
– What they mean by custom reporting?
[00:21:08.06]
– I think this is about the custom, customized reporting within the product. And how a little bit more detail they’re seeking on that.
[00:21:20.08]
– So that’s still being discussed. And so no specific details yet, but our goal is that the client can take this record and send directly to his client, their clients.
[00:21:36.09]
So that will be, there will be some kind of white labeling that you can add your logo, maybe change your colors, or it will be in the neutral colors that will fit your brand.
[00:21:54.07]
So this kind of customization, definitely there will be, maybe it will provide a way for people to just switch models they want to add to the report, for example, you want SPM, but you don’t want monitoring.
[00:22:13.09]
– Yeah. I can’t believe that we’ve got through all of our questions.
[00:22:19.03]
We could hang on the line if some of you have just been pondering, but haven’t typed your question in yet. We’re more than willing to do that.
[00:22:29.05]
I know this is an exciting product within the WP Engine platform. So, we’re ready.
[00:22:35.07]
– Yeah. We’re waiting for you.
[00:22:37.03]
– We’re ready.
[00:22:38.03]
Here’s one for you, Peter. Would you generally recommend Smart Plugin Manager for all types of sites?
[00:22:50.02]
– Yes. I mean, I don’t see any specific type of website that you shouldn’t use SPN.
[00:22:58.00]
If you’re using WordPress, and definitely you are, if you’re on this conference, then you have plugins there is no WordPress website without plugins, and you have to keep your plugins up to date.
[00:23:12.05]
So there’s no reason why you shouldn’t SPM.
[00:23:16.05]
– Perfect.
[00:23:18.09]
You know, I think I just want to, for fun, ask you a fun question before we close out. How are you staying sane and having fun during COVID?
[00:23:31.00]
– Oh, so that’s a really interesting one. I’m playing with, or building Lego blocks together with my sons. So that’s what we are doing together.
[00:23:44.09]
– How wonderful. That sounds great. New hobby.
[00:23:49.04]
– Yeah, new hobby.
[00:23:51.00]
– Yeah. See, once a builder, always a builder. You’re just expanding your horizons.
[00:23:55.03]
– Exactly.
[00:23:57.05]
– Yay.
[00:23:58.05]
Well, what a joy to talk to you from Austin to Krakow, like with a global audience. What time is it there right now?
[00:24:06.08]
– It’s almost 10:00 PM right now.
[00:24:10.03]
– Okay, look at this, amazing.
[00:24:14.01]
Well, Peter, thank you. I hope you all benefit from this Q&A session and we’ll go ahead–oh, hey, look, there’s another question before we sign off we will ask it, do you have to purchase multiple licenses for multiple Smart Plugin Manager environments?
[00:24:34.04]
– Yes. So one environment is one license. If you want to run SPM, for example, on production and staging, then you have to have two licenses.
[00:24:45.06]
But you have to think about that, that commercial plugins might not work in your staging environment. So it might occur that SPM will tell you that we have problems if you update your plugins on staging but it might work on the production because your domain is connected to your commercial license.
[00:25:11.05]
– Okay, great. Let’s see if we might have one more, I’m pausing. I think this might be a wrap everybody.
[00:25:25.00]
Nope. One more. How easy is it to roll back changes?
[00:25:32.04]
– So, SPM by default, rolls back any detected changes, but if, let’s say SPM will detect changes as insignificant and you don’t agree with that judgment and you want to roll it back,you can just go to our portal to the backup sections, and just run another roll back as you can do with any other background.
[00:26:04.00]
– Okay.
[00:26:06.00]
Great. Well, again, thank you everybody. Appreciate this. This session had a lot of great questions and I look forward to seeing you in the next breakout or whichever one you choose to hear next. Thanks again, Peter.
[00:26:23.04]
– Thank you.
