Press Release
John Verry (00:28.106)
Hey there, and welcome to yet another episode of the Virtual CISO Podcast. With you as always, your host, John Verri. And with me today, Scott Wasnicki. Hey Scott.
Scott Woznicki (00:41.559)
Yeah, thanks.
Scott Woznicki (00:53.637)
HM.
John Verry (00:56.932)
Ready for the weekend? I don’t know about you, but this has been a tough one for me.
Scott Woznicki (01:01.9)
Yeah, this time of year there’s always a lot going on.
John Verry (01:04.968)
Yeah, especially the holiday season. Although the people listen, this will probably get posted for four to six weeks or something like that. So but then they’ll be like, what do they mean this time of year? Alright, so I always I always like to start simple. Tell us a little bit about who you are and what is it that you do every day?
Scott Woznicki (01:13.42)
Yeah.
Scott Woznicki (01:20.868)
Yeah, so my name is Scott Wozniicki. I am the National SOC Practice Leader at CBiz MHM. I’ve been at CBiz for about three years now. Prior to CBiz, I spent the majority of my career, I’ll say, for three of the big five firms, just in various compliance roles, internal audit, IT audit, obviously SOC, et cetera.
John Verry (01:49.562)
So you guys did, that is in being CBiz MHM, did a relatively large and to me, an interesting survey on SOC attestations. I think that not only is there some interesting data in there, I think there’s some good insights for people that we can all learn from. So to set just a floor for the conversation, what is a SOC report and what is the difference between a SOC 1 and a SOC 2?
Scott Woznicki (02:16.996)
Yeah, so a SOC report is, it stands for Systems and Organization Controls. It is a review of controls around third party service providers. So getting into the specific differences between SOC 1 and SOC 2. SOC 1 is kind of the new standard or newer standard around
the old SAS 70, some people still refer to it as, but those are the SOC ones, look at controls around entities that are managing.
controls relevant to financial reporting. So kind of the most common example is an ADP who does payroll for thousands of companies around the globe. They don’t want every company that utilizes them to have their auditors knocking on their door saying, hey, we need to test your payroll controls. So ADP says, well, time out guys, we’re gonna hire one auditor, look at our controls holistically for managing payroll processes for all companies. And then what they do is they get their SOC one
disseminate that out to all their customers.
SOC 2 came about because when you started to have service providers that didn’t just manage financial reporting controls, these newfangled concepts such as cloud computing came up. The AICPA kind of took a step back and said, well, this wasn’t really the intent of a SOC report, at least a SOC 1. So what they did is they created a SOC 2, and that’s for service providers that may
Scott Woznicki (04:00.294)
controls relevant to financial reporting, but they do manage sensitive data. So again, kind of the most common example would be like an AWS. Companies that utilize AWS have a vested interest to say, Amazon, how do you protect our sensitive data?
John Verry (04:17.846)
Gotcha. Now for the purpose of this conversation, we’ll probably talk a little bit more about SOC 2, and just to kind of make sure we’re on the same page, we’ll probably be talking mostly about SOC 2 Type 2 Service Audit Report. So would you differentiate the difference between a SOC 2 Type 1 and a SOC 2 Type 2, please?
Scott Woznicki (04:35.108)
Yeah, so it gets confusing because you get SOC 2, type 2, SOC 2, type 1, SOC 1, type 2, etc. Way too many types of reports. But the difference between a type 1 and a type 2, a type 1 is less common because what it does is it provides an opinion only on the suitability of the design
Scott Woznicki (05:05.242)
Are your controls in operation? Are they operating effectively over a period of time? That’s a type 2 report.
John Verry (05:12.174)
Gotcha. Okay. And the SOC 2 Type 2, of course, because of the fact that the auditor is looking at an observation period, most typically a one-year observation period, that provides a higher degree of assurance, right? Not only is the design where we need to be, but we have a good validation of the ongoing operation of safe controls. Cool, perfect.
Scott Woznicki (05:31.88)
Correct.
I mean, the SOC 2, the relevancy for this conversation is probably just because you’re seeing more of an explosion around the demand around SOC 2. SOC 1’s been around for a large number of years. A lot of companies that manage financial controls have had a SOC 1 in place for a long period of time. But now all of a sudden, the question that’s rightfully being asked is, OK, companies that house or manage sensitive data, how are you
And so there’s been a huge increase in demand.
John Verry (06:07.306)
Yeah, it makes perspective. So your report validated, at least for me, from my observational experience, that the most frequent combination of the five trust services criteria within SOC 2, and the most common criteria that are used is security and availability, which is what we typically see. So from your perspective,
What do you, when do you see and recommend the other three services criteria?
Scott Woznicki (06:39.076)
Yeah, I mean, so just for those who aren’t familiar, I mean, the other three categories are confidentiality, processing integrity, and privacy.
I was actually kind of surprised that there weren’t more instances of just security only. That’s kind of the bare bones minimum because oftentimes when clients or service providers get a request for SOC 2, it’s not very specific in regards to what the requirement is. So I see a lot of companies actually that just kind of, if you’re just asking for a generic SOC 2, they only do security.
but that was only in like 15% of the SOC 2s that we looked at. I mean, the other ones, to your point, confidentiality and processing integrity were the next most common that we saw. Confidentiality at like 34%, processing integrity at 16%. Confidentiality, we usually recommend in instances if there is sensitive data.
there’s kind of a fine line in interpretation and understanding of confidentiality versus privacy. While people that kind of work in IT space kind of know the differences a bit more. Confidentiality is any data that a company has deemed confidential. It could be individual information specific to a person like PII, or it could be just anything like
John Verry (07:56.137)
Mm-hmm.
Scott Woznicki (08:19.85)
secret recipe or something like that. So it can have a much more general definition versus privacy, which is personally identifiable information. The reason we see more confidentiality, though, than privacy is I think companies want to show that they have programs in place to manage that sensitive data. But it’s an easier lift, if I’m honest. Yeah, I mean, you can look at it.
John Verry (08:44.342)
get much, much easier left. Privacy’s a bear.
Scott Woznicki (08:49.668)
Yeah, you can look at privacy and the number of areas that you have to cover. It’s almost equal to that of all the other four categories. So that’s why you don’t see a lot of it.
John Verry (09:01.558)
Yeah, and effectively, if you’re going, generally what we see is if you’re going to do privacy, the other attributes go along for the ride. So just do them as well. So when I think you had something like 5% or something or 4% were privacy, when you looked up the score, you knew that 5% of the next two up were also part of privacy, right? So realistically, that processing integrity almost never gets hit alone or without privacy. And I was surprised how confidentiality was.
Scott Woznicki (09:17.189)
Yeah.
John Verry (09:31.51)
So that’s interesting. So, and you’re right, I rarely see security alone. And you would think a lot of people, I think it’s because, you know, the availability isn’t really that much more of a lift if you’re gonna go through a formal process. And I think it provides a lot of value, you know, ensuring, you know, I mean, ensuring the availability of data, especially in today’s world where many of the threats that we face are around the availability of data. So I think that it kind of explains that. Another thing was cool about the report.
Scott Woznicki (09:42.065)
Yeah.
John Verry (10:00.574)
and we see the same thing in our work, is the variation on the control count. It was insanely wide. So, A, talk a little bit about how wide that was, and then if you could also touch on.
John Verry (10:17.51)
Why do you think we see that wide of a number of controls in the control matrix? Is it some people are just becoming insanely granular with their controls? Is it the complexity of certain systems and scopes? Is there something I’m not thinking of?
Scott Woznicki (10:35.12)
I mean the short answer yes all the above I Mean to get into more specifics. I mean It is interesting because sock 2 is a very prescriptive framework, right? You have the answers to the quiz This is the controls that you need to have in place And you just need to map out what you do internally to the framework. So You would think it would be
John Verry (10:39.971)
I’m sorry.
Scott Woznicki (11:04.7)
much more condensed in regards to kind of the range of controls. One thing I see in practice is, for lack of a better term, some overkill, which was kind of prevalent within the study that we did where you saw some instances of a very high number of security controls, for instance.
John Verry (11:26.254)
It was like 349 or something was the biggest one. I was like, I’m like, who the heck has three? Yeah, I would have looked to have seen it just to see how they, you know, like I can’t, I couldn’t think of 300. I mean, if you take like a really robust framework, like, you know, ISO, you know, I mean, like, you know, 114 in the old version, now we’re down to 90. Like, where are you going with 339 controls?
Scott Woznicki (11:30.648)
Yeah, I don’t even know if I can name that many controls.
Scott Woznicki (11:48.666)
Yeah. I mean, one thing I see in regards to where companies are probably doing more than they need to is under each of the criteria.
John Verry (11:50.634)
Ahem.
Scott Woznicki (11:59.92)
the ASCPA provides examples of how you can achieve those criteria and they call them points of focus. There’s, for whatever reason, this misunderstanding amongst some firms that they feel that they need to hit every single one of those points of focus. And so all of a sudden you’re testing way more than you need to do because you probably only need a fraction of those to actually achieve the criteria.
John Verry (12:24.142)
So what would you say, I would assume that you would say there was a danger to having too many controls.
Scott Woznicki (12:32.28)
I mean the danger is that you’re overextending your internal staff on having to do a lot of extra audit focused efforts and you’re probably overpaying for your report because if you think about what’s the cost driver on the SOC report it’s how many controls do you have.
John Verry (12:50.866)
Right. And then also, with every control, especially if they get either a little bit arcane and confusing, or there’s more of a chance that you’re going to miss something and that you’ve got a higher likelihood of having an exception noted, which is one of the things that we ideally don’t have in our SOC 2 reports, right? It denigrates the opinion of somebody reviewing that report a bit about the maturity of your thing. And then on the lower end…
Scott Woznicki (13:07.781)
Yep.
John Verry (13:20.202)
I think the number was like 39 or something of that nature. It was also really low. What was going on there and do you see the same challenges in having too few controls?
Scott Woznicki (13:32.885)
Yeah, eh.
I mean, when there’s, I forget how many, 33, 36 different criteria for security, and you only have that many controls, I mean, you’re averaging about one control per criteria, and the risk that you run there is that if that one control fails, the criteria is going to fail. And when you provide an opinion and a SOC report, you’re basically specifying if the criteria’s individual criteria were achieved.
So you run a high risk factor there.
John Verry (14:06.658)
Gotcha. So just generally, if you were talking about security and availability, right, which was, I think, 70% of the reports, you know, do you kind of see a sweet spot if somebody just was, you’re having a beer with someone who was implementing a SOC 2 environment and said, how many controls do I have in my control matrix, security and availability? Would that number be 50, 70, 100, 200?
Scott Woznicki (14:28.956)
I mean, a sweet spot for security is probably 50 to 60. And then for availability, you’re probably talking about half a dozen controls.
John Verry (14:42.11)
Okay, yeah, so there was definitely some serious overkill in some cases. So another thing which is I think a very confusing component of SOC 2s for people that don’t do this for a living is the concept of a subservice provider. So in the study you said that 95%
Most of the reports cited two or more subservice providers and 95% of those being carved out. So a couple quick things. Maybe touch on what a subservice organization is to make sure we have a clear understanding of that. And then can you explain what a carve out is and what are the implications to someone who’s, very often people review SOC 2s as part of the vendor risk management process. What are the implications to someone who’s receiving a SOC 2 if they see a carved out subservice organization?
Scott Woznicki (15:34.236)
Yeah.
So subservice organization is an organization that performs controls on behalf of your company So again, if you are managing or have created a SAS platform System that offers some sort of service Often you are utilizing some sort of cloud hosting company like an azure AWS etc. So in those cases
or Azure would be a subservice provider.
carve out portion is basically stating that you are not testing the controls of that subservice provider. So they are managing some element of your control environment on your behalf, but you are carving out from this SOC report that portion of the audit. And the reason being is, I mean, if you think about it,
what vested interest would AWS have on wanting to be included on every single one of their customers’ SOC reports? I mean, that would be just insane to be able to do. I mean, if you could convince them to do that, that would be kind of like being able to sell hair products to Patrick Stewart or something.
Scott Woznicki (17:01.808)
The only time that you, so I mean with the number of companies that utilize subservice providers, majority are going to utilize the carve out methodology. The only time that we ever see usually the inclusive model is when
If you think about if you have like a parent-child relationship where it’s really kind of the same organization and the parent or company probably has some entity level controls that they manage on behalf of the subsidiary, those are the sorts of instances where everyone’s going to have a vested interest to kind of play nice and participate within the scope of the audit.
John Verry (17:38.358)
Gotcha. Now, just to be clear, when you said that you’re not validating the operation of those controls, you’re referring to the SOC 2 attestation firm is not. Does the client under a SOC 2 have a requirement to validate that those controls are running effectively, if they’re reliant on them for the security of the system?
Scott Woznicki (18:09.768)
Sorry, I might need you to repeat that one. Yeah.
John Verry (18:11.406)
So think of vendor risk management, right? So all I’m saying is that, you know, you said, hey, they’re not validating those controls. I think you meant that the auditor is not validating those controls. The SOC 2 auditor is not. So does the SOC 2 auditor though, so look to validate that the entity being audited, receiving the SOC 2 certification, has some mechanism to validate that third party, the carved out organization, has implemented those controls and is operating them effectively.
Scott Woznicki (18:22.064)
Yes, yes, correct.
Scott Woznicki (18:40.872)
Oh, yes, absolutely. That is actually part of the SOC 2 framework around vendor risk management. How do you obtain assurance around any third party service providers? Are you reviewing their SOC reports, et cetera? It’s just the fact that the opinion itself, like you said, from the service auditor’s perspective, they were not independently also auditing that Carpelt entity.
John Verry (19:06.882)
The reason I specifically brought that up is I have actually had clients tell me, oh no, we don’t need to do that. They’re going to be a carve out. And I’m like, no, that’s not exactly how carve out works. So.
Scott Woznicki (19:19.656)
Yeah. Or I always love the one about no, we only use reputable firms. It’s okay. We don’t we don’t need to review it. But then I’d with
John Verry (19:26.806)
Yeah, exactly. And listen, and I do feel bad for people, right? Because I mean, there’s like, you know, Microsoft or AWS are not gonna answer your security questionnaires, you know. But, you know, the best we can do is we can go to their pages that they post and we can download the most recent copies of all the third party attestations. And when you walk in the door and you say, okay, you’re hosting, how do we know the host is secure? Hey, this is the best I can do. And you’re gonna look at that and go, yep, that’s pretty good. I can accept that. Okay.
Let’s see what else we got here. The other thing, this one really shocked me. Only 8% of the organization’s surveys used internal audits as part of their SOC 2 testing approach. I was like, holy cow, I thought most orgs did some form of internal readiness assessment. So why do you think that is?
Scott Woznicki (20:09.882)
Yeah.
Scott Woznicki (20:20.124)
Um, I mean, I was also kind of shocked by it too. And as a formal internal audit director, it kind of crushed me as well. Um, but I think it’s, yeah.
John Verry (20:30.685)
It hurt your soul.
Scott Woznicki (20:33.96)
because internal audit always wants to feel validated and that we’re helping out the organization for which we work. But I think it’s kind of twofold. One is perhaps the internal audit organization didn’t have the bodies to be able to support the SOC audit. I mean, a lot of shops are one to bed.
John Verry (22:23.638)
Another thing that I found really surprising was that only 8% of the organizations actually used internal audits as part of their SOC 2 sassing approach. Why do you think that is?
Scott Woznicki (22:54.148)
I mean, for starters, I mean, as a formal internal director myself, I was kind of crushed to see that fact as well. As internal auditors, we always want to feel validated in that we’re doing our best to help the organization for which we work. I mean, if I take a step back and look at it from the service auditor’s perspective, I think it’s probably twofold. One is…
Did internal audit have the resources to potentially be able to help and or support via like a staff augmentation role or something like that? A lot of internal audit shops are probably like only a one or two man shop anyways. So to be able to have extra bodies to be able to dedicate to assist on the SOC audit is probably a big ask in regards to just taking up an added time from their internal audit program.
The other thing if I’m honest is probably also just around kind of the overhead that Audit firms have instilled to rely upon the work of others. There’s a lot of standards and requirements around certain levels of re-performance that must be done and
levels sometimes get so onerous that you’re almost better off testing something independently yourself. So it’s probably a little bit of both.
John Verry (24:23.402)
Yeah, it is an interesting. It’s an interesting differentiation between ISO 27001 and SOC 2, right? Because part of ISO 27001 is the requirement to conduct an internal audit. And under ISO, the idea behind that is that it’s a mechanism by which you validate the effectiveness of the information security management system. And then really, the audit is intended to be just a validation that you’re actually following these processes. So it’s kind of like a different approach.
Scott Woznicki (24:50.33)
Yeah.
John Verry (24:54.747)
So I was really surprised by it because ISO, and I’m a fan of ISO, so I’m allowed to say this, has a little more slack in it, I think. I think because the fact that they’re validating the operation of the management system, not necessarily directly validating the operation of each of the individual controls, if you miss something in an ISO audit, excuse me, if you miss something, an ISO audit is less likely to identify it, where because…
in a SOC 2 audit, we are so extensively testing the controls. If you miss something, it’s going to be identified. So in a weird way, it would make SOC 2 more important to have some form of independent validation ahead of the external audit, but yet that wasn’t the case.
Scott Woznicki (25:31.983)
Yeah.
Scott Woznicki (25:42.596)
Yeah. Yep, I agree with you.
John Verry (25:46.562)
So I was about to say I’m also surprised about the number of exceptions noted, but maybe I shouldn’t be in light of the fact that what we just went through. If they’re not doing readiness assessments or internal audits, whatever you want to refer to them as, yeah, maybe it isn’t that surprising that you guys noted that the average SOC 2 report had 2.7 exceptions. My guess would be that the median would be lower. And the reason that I…
Scott Woznicki (25:56.389)
and.
John Verry (26:15.438)
guess that, I’d be curious if you guys looked at that, is because we do a lot of third party risk management where we’ll operate third party risk management programs on people’s behalf, we review a lot of SOC 2 reports. And I would say it’s unusual for us to see a report with more than two exceptions. So my suspicion is that there’s some reports, and I think you might have even had data that supported this where there are some reports that have a lot, lot of exceptions and they might drag that average up. Is that fair?
Scott Woznicki (26:42.8)
Yeah.
Yeah, I mean, there were instances of 11 plus exceptions and that probably did drive up that average.
I mean, I kind of facetiously look at it when I see reports on a consistent basis that always have no exceptions noted, then I’m kind of skeptically saying, all right, well, how deeper are you looking? Because I would expect some exceptions periodically. I mean, it’s human nature. We’re going to err. So people fret when they have an exception. But it’s…
just kind of acknowledgement again that there’s this human element to controls. And I mean, it’s not necessarily a bad thing to eventually just kind of be able to take a step back and say, okay, we have opportunity here.
John Verry (27:40.394)
Yeah, I would say that I agree with you, by the way. And, you know, I’m a fan, like, we’re ISO certified as a company, and I want every nonconformity that you can find. And I will give them to you if I can figure that, find them, right? Because I act as our CISO, for CBIS, Pivot Point Security, not CBIS. And, you know, because look, if we’re gonna spend 30,000 bucks or something of that nature, I wanna get some value out of it.
Plus, I’d rather find out this way than find out because I get notification from some third-party hacker that they’re in our systems and they want money. So I agree with you. I think people should be more open to them. I think one of the challenges that you have specifically with SOC 2 is that because it’s such a good third-party attestation, people use it to prove to their clients that they’re secure. And I think that most clients, when they receive a SOC 2 Type 2 Service Orders Report, they
Scott Woznicki (28:19.356)
Yeah.
John Verry (28:37.498)
and you start to get to three or four exceptions, it starts to actually make you a little bit nervous. So I think that’s kind of one of the inherent challenges that we have here, is that we want them to be open and accepting of these, but we know that if they’re handing this report off and they have too many, it might actually negatively impact their client’s perception.
Scott Woznicki (29:00.324)
Yeah, absolutely. I mean, service providers have the opportunity in section five to provide responses to any exceptions that were noted. That being said, I mean, you do have to kind of take a step back at a certain point, and if you start to see a more pervasive pattern of control exceptions, kind of ask, okay, what is the oversight that’s?
ongoing air is it sufficient?
John Verry (29:31.266)
You know, I’m embarrassed to cite that I did not realize that Section 5, that you could do that. And I don’t recall seeing it very often. I don’t recall reviewing too many reports these days, but I don’t really, I don’t think I’ve noted that section used very often. How often will people, in your experience, use Section 5 to explain something? Or if there’s just one or two, do they not take the time?
Scott Woznicki (29:55.956)
I feel like it’s…
pretty prevalent in majority of instances when there are exceptions. I know from our perspective, we always encourage clients just in anticipation of receiving questions from their customers that they should put in some sort of details in regards to what have you done to remediate the issue since it has been identified because that often alleviates a lot of follow-up that they’re going to get in response to those exceptions.
John Verry (30:26.558)
Yeah, I think one of the reasons I probably have never noticed it is I have a tendency to read the system description and then I have a tendency to just search on exceptions noted and control off my way through and just…
Scott Woznicki (30:38.296)
And that’s why we did the study so that people know that there’s a lot of other kind of hidden facts to consider.
John Verry (30:42.638)
I’m not all kidding aside because I’m thinking myself. Why have I never noticed that? And the answer is, it’s just exactly that. Is that in which so thank you. I’m an idiot. I will definitely now look for section five because if I do see an exceptions note, at least I’m going to have a little bit of a an expository.
So the other thing I think, you know, kind of maybe in second place to the carve out thing that I think people are sometimes a little confused about is the concept versus an unqualified opinion versus qualified opinion. And you had 11 of the 154 reports you reviewed had unqualified opinions. And that was despite as many as 11 exceptions on one of those reports, which almost sounds a little bit concerning.
Scott Woznicki (31:24.956)
Yeah.
John Verry (31:28.206)
Can you talk about qualified versus unqualified and your thoughts on that particular issue that I raised?
Scott Woznicki (31:28.811)
indeed.
Scott Woznicki (31:35.32)
Yeah, I mean in layman’s terms unqualified is the good one, qualified is the bad one. More formally, unqualified means that the controls were sufficiently designed and operated effectively over the period of the audit and also that the system description was sufficient.
Qualified means that there were either control objectives and or trust service criteria that were not sufficiently achieved based upon exceptions being noted.
John Verry (32:12.11)
So oversimplified pass-fail.
Scott Woznicki (32:16.829)
Yes.
John Verry (32:19.05)
And then if someone had 11 exceptions and it was still an unqualified opinion, to me, is that somebody trying to maintain a customer instead of?
Okay.
Scott Woznicki (32:37.092)
Yeah, yeah.
I mean, I hate to say it, but that’s kind of what it feels like. I mean, if I were to get that report, I know a lot of people when they get socked to is kind of do like you said. I mean, they just kind of read through what’s the opinion, what were the exceptions and if it’s unqualified at the end of the day, they’re okay. Um, but they don’t always take a step back and kind of independently ask kind of those cynical questions of.
Okay, is there something here that I should be concerned about or maybe be thinking through?
John Verry (33:17.342)
Yeah, no, you know, if it were me doing a vendor risk management, if a customer asked me about a critical supplier that had 11. 11 exceptions noted on the SOC 2 report, I would express some concern, you know, and one same page there. So you mentioned the system description and, you know, system descriptions are interesting to me. I would argue that there are not science.
And the range and what’s included in the link is pretty remarkable and your study cited that. I can’t remember what the numbers were, but it was crazy. It was like 20 pages to 200 pages. I mean, it was nuts. So how would you give guidance to someone on writing a good system description, right? Obviously we want it to be just as long as it needs to be and include just what it needs to include. How would you go about guiding somebody on that?
Scott Woznicki (34:11.852)
Yeah, I mean, you kind of stole my thunder there. I mean, I think it needs to be succinct but informative.
If you start to get into system descriptions that are 100 plus pages, you have to kind of ask yourself the question, I mean, how many people are going to actually take the time to read this?
feels like a bit of overkill. But then there’s other ones. I think one of the reports was only about six pages. And you have to ask yourself, I mean, how did that really sufficiently achieve the objective of what’s called the fairness of presentation, which is one of the aspects of the sock auditor’s opinion specifying if the description sufficiently describes the control environment.
Scott Woznicki (35:04.982)
reading through the system description and doing a mapping to all the different control aspects that were being tested and I don’t know how you can do that in six pages if the average number of controls is 60-70 that’s about almost a sentence per control so getting back to the point I mean I think that it should cover all the key elements of the control environment
John Verry (35:20.927)
Yeah.
Scott Woznicki (35:33.617)
but doing it in a succinct and clear manner.
John Verry (35:38.042)
So you made me think of something I never thought of before, which means that, so I’m getting what I like out of these podcasts is when I learn something or make a connection I’ve never made before. So someone gives you a 200 page system description and they have five flowery pages on the implementation of their SIM and all these wonderful things that it does. Does the auditor conducting the audit…
on behalf of the CPA firm, have any obligation to validate that system description and every statement made in there is accurate, right? So do you have to literally go through that and then look at the control matrix? And if somebody said that they use artificial intelligently enabled counter-threat intelligence, so-and-so, do you actually have to have an obligation to look for a control of that nature to validate that statement is acceptable in the system description?
Scott Woznicki (36:36.836)
You’re supposed to. That’s why the control description should be very succinct. I mean, the description should not include extraneous information that the auditor is not able to independently validate. Which is…
Scott Woznicki (36:54.556)
why we kind of do that mapping exercise and there’s also looking for I think you use the term like flowery language of we’re best in class or the world’s best this the intent of these reports is not only for like marketing purposes here so you want to kind of keep it so that it’s factual based.
John Verry (37:21.262)
Will you often make suggestions or will you often, strong suggestions even mandate that somebody changes language in a system description if it’s, in your opinion, deceptive or is making a claim which can’t be substantiated by the work you do?
Scott Woznicki (37:39.191)
Um
Usually in those sorts of instances what we’ll do is we’ll recommend that we kind of carve out that section Of the description and move it to section 5 Section 5 is kind of that extraneous information again where customers are providing Responses to exceptions, but it’s also any other sorts of information that they want readers to be aware of However section 5 is not part of the audit opinion. So
John Verry (37:52.082)
Okay.
John Verry (37:56.109)
Okay.
Scott Woznicki (38:11.09)
we’re not under the obligation to be able to go and validate all that information. So if you want to include this extra information, often you see it around, like in a SOC one, around disaster recovery, things like that, that aren’t really financial reporting controls.
John Verry (38:11.115)
I got you.
John Verry (38:26.89)
Yeah, almost turning a SOC 1 into a SOC 2 without doing it, right? Hey, plus our security is this and our DR is this and our BCP is this. Uh, and it gives them an ability to market in there, but it doesn’t change the audit cost or the audits. Okay. That’s interesting. So, so that, the, that section five, you know, I wonder how many people reviewing a SOC 2 recognize that about section five. Uh, so that’s interesting to me.
Scott Woznicki (38:30.609)
Yeah.
Scott Woznicki (38:34.714)
Yeah.
Scott Woznicki (38:52.292)
Yeah, I mean there is a paragraph within the opinion that actually states the fact that we do not provide an opinion on section five.
John Verry (39:03.278)
Okay, I got you. And then, so just out of curiosity, if someone has a 200 page system description, something which should be 20 pages, would you charge them more for their report because you have more work to do?
Scott Woznicki (39:23.768)
Yeah, I mean it’s…
John Verry (39:25.378)
So it’s another good reason for people to be concise is because an auditor, you know, you’re paying for somebody to sit there and map that, which I would imagine is a lot of work. I mean, mapping, that’s got to be, that’s got to be days, many days for the work. That’s pretty cool. Again, things you don’t think about sometimes. So, so, you know, I would encourage anybody who is in the attestation business, vendor risk management, third party risk management.
Scott Woznicki (39:36.204)
Yeah, absolutely.
Scott Woznicki (39:42.29)
Yep.
John Verry (39:53.418)
anyone who’s thinking about getting a SOC to read your report. What are some of the other key takeaways from the report that you think would be cool to share?
Scott Woznicki (40:04.248)
Yeah, I mean, I think really kind of the intent of this benchmark study was a we noticed that there was kind of this gap within the sock world of not having a previous benchmark study. I’d never seen one.
done anywhere else. I mean you see it with Sarbanes-Oxley. A lot of firms do that but I’ve never seen one specific to SOC. And really the intent was not just to kind of give the generic here’s the number of controls, here’s the number of exceptions. It’s also point people in the direction of other potential like
red factors that they should be looking at. Like a good example might be around the duration to issue. That was one that is always kind of near and dear to me and that’s the duration to issue is you look at the timeline between the end of the reporting period and when the auditor’s opinion was provided. So.
Most audit firms usually target like a 45 to 60 day window. So if the report period end is 1231, and usually 45 to 60 days after that, the auditor typically will issue the report.
But there were some instances and there was one kind of egregious example that went as high as 535 days from the end of the period date. So at that point, you’re like a year and a half providing an opinion on something that happened a year and a half ago. Yeah.
John Verry (41:36.645)
I’m going to go ahead and turn it off.
So they issued toilet paper is basically what you’re suggesting.
Scott Woznicki (41:46.636)
It’s it’s stuff like that. I don’t know and I probably doubt that people really look at or ask the question Okay, what went wrong here guys? I mean it’s one of two things either the auditor Was unable to sufficiently staff?
report to be able to get you something timely and or There’s something going on at that company that they just cannot focus on Getting their sock to out the door and no matter what that opinion is I’m gonna have some serious questions and concerns because why does it take you that long to be able to pull together the information?
John Verry (42:23.33)
So question for you, I know that CPA firms go through third party, I think they call them quality audits each year, right? Where another firm comes in and looks at the work that you do, samples a couple things and validates it. When they come in to do that, would it just be dumb luck that they do or don’t grab that particular report?
or the unqualified opinion with 11 nonconformities, again, because ideally we would love somebody to be holding these CPA firms feet to the fire.
Scott Woznicki (43:00.856)
Yeah, agreed. I mean, so the process you’re talking about is a peer review. So audit firms kind of auditing each other to ensure a certain quality of work. I mean, it’s the intent is definitely there. The intent is good. I’m not sure it always works quite to the level that it was intended.
A lot of times peer reviews they’re looking at, high number of engagements over a very short period of time and so they might not be doing a super deep dive and also it’s probably dependent upon the relationship too. I mean if you have two smaller firms that maybe are kind of equal in their quality standards, they might piggyback off each other saying, hey I’ll do your peer review if you do mine.
That’s just kind of the nature of the beast, unfortunately.
John Verry (43:59.21)
Yeah, look, I mean, there’s no.
There’s no perfect attestations out there. There are no perfect auditors out there. There are no perfect control systems to ensure that the auditors are doing what they’re supposed to do out there. I think in general it works pretty darn well, but that doesn’t mean that there are not some, and it doesn’t matter what the attestation is, doesn’t matter what the standard is, there’s some people out there, and I think it’s incumbent upon you as the entity that is acquiring the third party attestation to ask the right questions, to know that you’re getting a quality opinion.
Scott Woznicki (44:04.476)
Yeah.
Scott Woznicki (44:31.716)
Yeah, absolutely.
John Verry (44:33.828)
Awesome. This has been fantastic. Is there anything we missed that you want to bring up?
Scott Woznicki (44:39.63)
Uh, no.
John Verry (44:41.892)
If somebody wanted to get a copy of this report or wanted to get in touch with you guys over there, what’s the best way to do that?
Scott Woznicki (44:49.896)
So the report is available on our MHM website. There’s a section on SOC reporting and you can download it from there. If you’re interested in reaching out directly, you can always reach me via my email address and or LinkedIn. It’s just my firstname.lostname at cbiz.com.
John Verry (45:12.318)
Excellent. Would it be okay if we put a link to it when we post this on our website? Awesome. Scott, this has been fun, man. Thank you.
Scott Woznicki (45:17.936)
Yeah, of course.
Scott Woznicki (45:22.448)
Yeah, thank you, John. Appreciate it.
