BARCObservabilityfor AlInnovationAdoption Trends, Requirements and Best PracticesBARC research studyResearchsponsoredby:CollibraJorecisely
AuthorsTimm Grosser|SeniorAnalystKevin Petrie|VPofResearch,Data&AnalyticsatBARCatBARCUSTImm Grosser isaSeniorAnalystData&KevinPetrieistheVPofResearchatBARCAnalyicsatBARCwithafoausondatastrate-whereheleads thedata managementprati-gy data govemanceand data management,ceand wrtesaboutpicsuch asAl datain,His core expertise is the definition and im-tegration and data governance.For30yearsKevinhasdeciphered whattechnology meansdemenation of data & analytis strategyto pradtitioners,as an industy analys. in-organization, architecture and software sestructor, marketer, services leader., and techection.journalist.Timm isapopular speakerat conferences and seminars andhas authoradKevin launched a data analytics services team forEMCPivotal in theAmeri-cas and EMEA, and ran field trainingat the data integration software providerAttunity (now part of Qlik).A frequent public speaker and co-author of twobooks a boutdata management, Kevin most lovesteaching data and Alleadersabaut evolving strategies, tools and techniques to capitalize on the value ofdata.ContactContactMail terosser@barc.com.Mail:kpetrie@barc.comwww.barc.comSocialMedia:SocialMedia:linkedin.com/timm-grosserlinkedin.com/kevinpetrietech
Table of ContentsExecutiveSummar.................Recommendation.......SurveyResul....Methodology127StateofObservabiiy.....9Compary-r..............How toTackleChallengesin Data@nbsenvabil.....1-AboutBARC...........30Growing Importance ofObserving Unstructured .at....1The RoleofObsenvablityT..--SoonsorProle....31Teamworkmatter......一..20AboutCollibra........32TheNextWave-DoesGenAlChangetheGame?..22AhoutPrecisel........33
ForewordAs artificial intelligence (Al) raises the risks and rewards of anay We conducted this survey with a wide range of respondents, fromIvtics, organizations recognize the imperative for transparentdata engineers and data scientists to CxOs and business processtrustworthy inputs and outputs. So there is no better time forthisowners. Our population includes stakeholders from a variety of report, which surveys 264 data and Al stakeholders across indus.-business functions and industries, most notably IT and manufac.tries about why, where and how they implement observability.turing respectively. Company sizes are evenly distributed; nearly1/3 have fewer than 500 emplovees and nearly 1/3 have 5.000 We examine three distinct observability disciplines: data quality,or more. We also representNorthAmerica and Europe in equaldatapipeline andAV/MLmodel.Ineachcase observabilitvreferstoportions, with a smallfraction ofrespondents comingfromAsialthe measurement. monitoring and optimization of theseelements.We find that most organizations now have formalized programsAs always, you can counton BARC toidentify and exolain mean.for data, pipeline and model observability. Organizations priari-ingful differences between these varioussubgroupstize privacy, auditability, and compli...
