snap-analytics-logo
Book a Call
snap-analytics-logo

Tag: Big Data

Be a Data Hero and deliver Net Zero!

Posted on by David Rice

The biggest problem in the WORLD!

It is clear that we need radical changes to save our planet. Governments, the private sector and individuals aspire to achieve ‘Net Zero’ – but radically changing the way we operate is not going to be easy.

Achieving this goal is going to be a huge challenge for big, complex organisations.  There are so many areas to explore, from reducing travel and fossil fuel consumption, leveraging renewable energy, improving efficiency of existing equipment, or simple behavior change.  With so much complexity the task can be daunting. 

Can data save us?…

Starting with data can help you to understand where the quickest and biggest wins are.  This helps you to understand what to focus on first.  As Peter Drucker once famously said “You can’t manage what you don’t measure”.

To create a link between desired outcomes and measurable targets you can use a ‘Data Value Map’. Whilst I love technology and data…it’s only useful when it drives actions and creates positive change.  The Data Value Map helps to visualise how data can help you to achieve your goals.  If your goal is Net Zero…it could look something like this:

Data Value Maps can be achieved using a mind mapping or collaboration tool (I like Mindmeister and Miro) and are best done as a highly collaborative team workshop…don’t forget to bring the coffee and cakes!

Now you have a clear view what data is required to measure and act (your “use cases”) to deliver the Net Zero goal.  Next you can score these in terms of Value and Complexity.  Something like a prioritisation matrix can help:

By focusing in on the ‘high priority’ and ‘low complexity’ use cases you can deliver quick wins to the business.  This will help you to demonstrate you are a true Data Hero and can help your organisation to fly!

Once you have prioritised your use cases, you can start to map out the underpinning systems and processes that are needed to deliver connected, structured data to drive your Net Zero goals. 

Delivering at lightning speed…

There are numerous technologies out there that can help you connect all of this data, but we love Matillion for being able to easily and quickly connect to almost any source and transform and join data to make it useful.  As a data platform Snowflake is fantastic for virtually unlimited storage, blistering speed, data warehousing and data science capabilities.  These technologies will certainly enable you to hone your capabilities as a true Data Hero!! There are also many other fantastic cloud solutions that can help you to supercharge your Net Zero data capabilities.

Join the Data League!

Snap Analytics’ team of Data Heroes are helping one of the UK’s largest food manufacturers to leverage data to drive positive change…but if we’re going to solve humanity’s greatest threat…it’s going to take a whole Justice League of Data Heroes.  So join us on this mission to save the planet, and lets all make sure the decision makers in our organisations have the data they need to drive positive change.  Don’t delay…be a Data Hero today!

We believe that businesses have a responsibility to look after our earth…it’s the only one we have!  We will give any organisation a 15% discount on our standard rates for any work directly linked to making a positive change to the environment!

The four pillars of a data strategy

Posted on by David Rice

Mention data to some executives and you’ll get a range of reactions, ranging from evangelical to utterly perplexed. The latter group might know data is important to their business and understand that using it correctly can bring significant wins, but there is often considerable confusion over how to best implement it. Fret no more, as here are the four pillars you need to launch a successful data strategy. 

People 

First look around your organisation and assess existing strengths and weaknesses. Do your staff understand the power and value of data? You don’t necessarily have to run out and employ a team of experts, instead audit existing skills. If somebody is a whizz with an Excel spreadsheet, chances are you can train them up to become more data literate.   

Next take a step back and look at the structure of your organisation. Is there a divide between your IT department and your business people? If so, that needs rectifying. To get the most out of your data, everyone needs to work together. Rather than make it the sole preserve of techies, a joined up approach will spread the responsibility and offer the best chance of creating a successful data culture. 

Process 

One of the quickest wins is in the optimisation of existing business processes. One company we worked with wanted to create a more efficient system for posting journals in their finance system. At the time they had a 20 step process for this seemingly straightforward procedure; laboriously copying information into Excel, running four different reports, then ticking off a long paper checklist before finally posting. Through automation we were able to reduce that process down to two or three steps, while still allowing for all important checks to take place. 

Ensure automation is built into your data systems. It will transform the working lives of even the most data-phobic employees, providing alerts on anything from fluctuations in sales to customer complaints. Rather than finding out about this in the annual report or having to dig through the data for answers, a simple warning symbol in your regular reporting will alert you to potential problems before they cause too much damage.    

Data 

While it’s important to have everyone on board, it is crucial to have trusted individuals overseeing data in areas such as customer and product information. You need people who can own that data and be responsible for it. Ensure that they have clear strategies around managing data quality and data governance. This stuff is as important to growing your business as your staff and product – treat it with the same care and you’ll reap the rewards. 

Technology 

Once you have thought through all areas of your strategy, only then should you commit to spending money on the necessary tech. We often meet companies who have muddled through by bolting on extra systems here and there. What they should have done is to step back and ask whether it might be simpler and more efficient to just start from scratch? Often building on a legacy system is a false economy, while investing in the correct, modern system for your needs will save you time and money. 

If you take just one thing away from this, it’s that upfront thinking is absolutely crucial. We see so many companies who have taken bad advice and invested in a pricey data lake or warehouse that ultimately didn’t serve their needs. They then have to start over, or are forced to work with compromised systems. Never overlook the importance of starting with a defined data strategy, it is one of the most important business decisions you will ever make. 

What better way to kick of you data strategy that a FREE 90 minute Data Strategy workshop tailored to your business? Book your call with one of our data gurus now to understand how data can help you to achieve your goals:

https://meetings.hubspot.com/david-rice2/free-90-minute-data-strategy-call

Lakehouses For Sale

Posted on by Tom Bruce

Whilst a ‘Lakehouse’ might sound like something out of a Nordic real estate brochure, we’re actually referring to ‘Data Lakehouses’ – the emerging technology in analytics.

The evolution of the Data Lakehouse all started with Data Warehouses which have been around since the 70’s when Inmon coined the term ‘Data Warehouse’. The Data Warehouse was designed to store numerous disparate data sets within a single repository ensuring that there is a single version of the truth for organisations. These data warehouse solutions are highly structured and governed, but typically they became extremely difficult to change and started to struggle when the volumes of data became too large.

Technology continued to evolve, and organisations started to have an exponential increase in the amount of data that they were creating as a result. This data that was being created by new applications was frequently produced in a semi-structured or unstructured format in continuous streams which is very different to the structured data from transactional systems that data warehouses were used to dealing with. For example, the Internet of Things (IoT) has meant that there are a huge number of devices that are continually creating sensory data such as smart draught pumps in pubs which automatically order new beer deliveries when the keg starts to run out!

This change in data meant that organisations had to adapt their data solutions. Around 10 years ago organisations started to create Data Lakes built using platforms such as Hadoop. Whereas Data Warehouses typically would only store the data that was needed for reporting in a highly structured format, Data Lakes were able to store all data possible but in its raw unstructured or semi-structured format. However, this lack of data quality and consistency often means that it is very difficult to use this data effectively for reporting which has seen much of the promise of Data Lakes unable to be realised (see our blog on ‘Data Swamps’). In addition, these Data Lakes, due to their lack of consistency, also struggled with a lack of governance and functionality compared to Data Warehouse solutions. To deal with this dilemma, lots of organisations typically have a data landscape with a data lake in combination with several data warehouses to help them meet their analytics needs. These extra systems have made the data landscape unnecessarily complex, with data frequently having to be copied between the data lake and the data warehouse.

Evolution of data storage, from data warehouses to data lakes to lakehouses
Figure 1 – Evolution of Data Solutions as Proposed by Databricks

Enter the ‘Data Lakehouse’ – technologies which provide both the benefits from traditional Data Warehouses in combination with the huge amounts of cheap storage utilised for Data Lakes which has only been possible with the advancement of cloud technology. There are several key features of a Data Lakehouse, however perhaps the most important of these is the ability to natively support both structured and semi-structured data. Snowflake, for example, has a native ‘Variant’ data type which means that you can load semi-structured data such as JSON, XML and Avro straight into the Data Lakehouse, whilst also providing SQL extensions to query this semi-structured data directly. The result is that it’s easy to store fully ACID compliant transactions commonly used in management reporting solutions alongside semi-structured data provided by streaming applications. Data Lakehouses also provide the auditability and governance that was clearly lacking in Data Lake solutions supporting traditional modelling techniques such as Star and Snowflake schemas. Lastly, by storing all of their data in one central data hub it reduces both the complexity of their technical landscape as well as reducing the cost as they don’t need to be storing data in both the data warehouse and the data lake (for more details, see our blog ‘Better, Faster, Cheaper”).

Data Lakehouses are the start of a really exciting new era in the world of data and analytics, offering a huge competitive advantage to organisations. Companies can finally use the huge amounts of data that they have started to create, being able to effectively use it in analytics and reporting to drive new insights and ultimately take better decisions.

5 things to do before starting a data project

Posted on by David Rice

You’re about to start a big data project. Fantastic! We’re big believers in the fact that every business can gain a real competitive advantage through analysing their data. It’s why we do what we do. 

But just before you go running off all excited, stop for a moment. If you really want your data project to be a success, you need to think about five key things before you even start. 

Understand the problem that you are trying to solve 

Chances are you’re looking to data analytics to fix a specific need, something which is causing inefficiencies and costing you money. Don’t assume though that by shaking the data tree enough times a solution will magically fall into your lap. First you need to look at your existing systems to see what exactly needs fixing. It’s only once you have a clearly defined vision and end point in mind, that we can see exactly how we can help. 

Define what success will look like  

Having identified what you want, it is time to think about what a successful outcome might look like. It helps nobody to embark on a data project without setting any specific goals or measurable outcomes. So make a plan, draw up a list of milestones, devise ways of measuring what’s happening and then track the results against that. One useful approach we’ve found is to run a user survey six months down the line and find out how people are using, or benefitting from the findings.  

Align with company strategy 

It’s all very well you dreaming up fantastic, innovative data driven projects that will change the very fabric of your business and the world generally. But it might be best, first of all, to check that your goals are something that fall in line the wider strategic direction of the business. Is this a problem you should even be solving? Is it a business priority? Will it help tick some important boxes when the annual report comes round? If the answer is yes to all the above, fantastic – you’re on your way to getting managerial buy-in and tapping up a healthy budget for an important piece of work. 

Data for the people 

You’ve addressed the needs of the bigger cheeses but don’t forget about the little guys, the people on the front line who are working hard to produce this data in the first place. Think about how this is going to benefit them in the long term, how will it make their day-to-work work easier, more efficient, or more effective? This is particularly pertinent if your business is going through a restructuring process. We’re great believers in the power of data analysis, but if you’re losing half your team it might not be perceived as the best use of company resources. 

Build the right team 

One commonly held assumption we come across is the idea that data is purely a tech led process: you identify a problem or need and the nerds crunch the numbers. It’s not that simple of course. To produce an effective outcome, you need quality input from people on the business side, members of the team who can provide insights into how the company works and what its goals and strategies are. You should bring together people who use the data in different ways and can provide the broadest possible range of experience. That way the insights we produce will be deeper, richer and ultimately more valuable. 

How to get buy-in for your data project

Posted on by David Rice

Despite the talk of a data driven revolution, the reality for many companies often lags someway behind the ideal of a business built on reliable detailed information analysed using AI. According to a Mckinsey Digital report on leadership and analytics, CEOs cite their biggest challenges to investing in data are “uncertainty over which actions should be taken” and “lack of financial resources.” Fundamentally, it appears that some business leaders still don’t believe that analytics have a high enough ROI.  

If you know that a data project could deliver huge value for your business but you’re struggling to get anyone else to appreciate the importance of all that expensive tech and numbers nonsense, here are a few ways to help change their mind. 

Speak their language 

There’s no point trying to convert anyone into data evangelists at this stage, instead work within the parameters of your organisation. Align your project to existing business priorities and show that you understand the CEO’s strategies. The best way to do this is to create a link between the results of your project and the financial benefits. Get those graphs at the ready! 

Remember that this is a business transaction and not a technical pitch. You’re wasting your time if you can’t convincingly demonstrate that you are addressing a particular business need, so resist using too many technical terms. Instead, explain using visuals which demonstrate that the outcomes from your data project align with the strategic objectives of the business and will bring tangible benefit to the company and the individuals who work there. 

You may need to convince people that you are addressing a demand that they didn’t even know they had. By the end you want not just buy in, but for them to believe that it was their idea all along! 

Recruit key players 

It’s not enough to expect the techies to wave a magic wand and sprinkle stardust over the business. Successful data projects are a partnership between the project team and key business users who work with the numbers on a daily basis. Don’t expect that a diktat from the 17th floor is going to be enough to drive them into making this thing a success, it’s your job to involve as many key players as possible, make them feel they’re being listened to and that they have some control over the direction of the project. 

Throughout the course of any data project, key business users should have frequent and regular opportunities to provide feedback. This agile approach will help to ensure that the solution is actually what the business wants – unless they are able to see the solution in action it’s impossible for them to really know what they want! An added bonus is that they will feel that they have shaped the project and will have a vested interest in the outcome. 

Recoup your investment 

From the C-Suite down to the office floor, what everybody wants is something that makes their work easier and the business more successful. If you can demonstrate that what you are doing is going to achieve tangible results they will sit up and listen. For example, a report which currently takes 5 hours will take 1 hour as a result of this project. And if that report is created by 100 people at a cost of £50 per hour, the bill drops from £25,000 to £5,000. 

To hammer that point home, McKinsey carried out analysis over a five year period which showed that companies who put data at the heart of their operation enjoyed marked improvements across all departments, with sales and marketing ROI increasing by 15%-20%. Investing in creating a data driven culture is vital for the growth of any business determined to stay ahead of the pack. 

Modern Data Warehouse – Snowflake & Fivetran

Posted on by David Rice

Why AI is a lot like teenage sex – and how you can get better at it

Posted on by David Rice

In 2013 Dan Ariely, professor of behavioural economics at Duke University, got the analytics world all aquiver when he stated: “Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.” His risqué comments appeared in the midst of a data revolution, the experience of which was less than satisfactory for many companies.

Six years on and we’re doing it better. However, AI is in danger of being in the same position as Big Data was six years ago. Some data strategists would have you believe that AI and its bedfellow machine learning are better than sex. But as with anything in the data revolution, don’t expect to achieve total fulfilment overnight.

Cost vs kudos

A report from the McKinsey Global Institute predicts that early adopters of this technology could grow the value of their business by 120%. It adds that those who fail to jump on board the AI gravy train could lose a fifth of their cash flow. No surprise that companies are throwing money at the problem – but not necessarily always for the right reasons.

Kudos is not enough of a reward for a business to pump considerable sums of money into an AI project. A survey by the US analytics company Figure Eight showed that the majority of companies were spending at the very least $50,000, rising to over $5 million for those serious about making it a central part of their business.

AI has now been around for two decades. Advances in tech plus heavy investment from the likes of Google mean that the cost of leveraging AI tools will continue to fall, levelling the playing field and allowing smaller businesses to utilise AI. If you’ve made it your business to amass a wealth of clean, properly managed data you are already well positioned to launch an effective AI project.

Let’s not get ahead of ourselves

With any data project, there are things you need to think about before you get started. If you’re looking to get started with AI specifically, first consider whether the problem you would like to address is best served by the technology. Don’t expect AI to act as a sort of panacea; you need to be deploying it in the right way and for the right reasons. If you’re unsure, talk to an expert first (yes, we can help with that) to assess what sort of data analysis would be best for your particular problem.

Maybe there is an area of your business you are certain would benefit from an AI solution – if only you could convince the CFO to invest. If they’re keeping a tight hold of the purse strings ask yourself: does this align with the greater corporate strategy? If no, you’d probably be better focusing your efforts somewhere that does.

Finally, when you have identified the right AI project and hired yourself a crack analytics specialist (hello), don’t assume that the thing will just run itself. AI is smart but it still needs help. That means putting together the right team – and not just a couple of people borrowed from the IT department. Successful AI needs buy-in from people who understand the business need and who are working with the numbers on a daily basis.

Get it right and you’ll transform your AI experience from a meaningless one night stand to a satisfying relationship that grows into something really special.

© Snap Analytics 2024

Site Map

Privacy Policy

Terms & Conditions

Sign up for news, insights & opportunities