Tidying Up your Infrastructure with Marie Kondo

Special thanks to Karl Widmer for sharing this image on Twitter that gave me the idea for this post. Check out his AWESOME blog HERE.

In this post, I’m going to show you some of the way’s you can apply Marie Kondo’s 6 basic rules to clean up your environment using Veeam ONE.

Who is Marie Kondo?

Marie Kondo is an Organizing Consultant who has her own Netflix show “Tidying Up with Marie Kondo”. The reality television series was released in 2019 and inspired many people to clean up the clutter in their lives. Her famous phrase is “Does it spark joy? If it does, keep it.

What is Veeam ONE?

It provides agentless real-time monitoring of your backup and/or infrastructure. It can also notify you before certain critical issues are about to hit your environment. Veeam ONE consists of:
• Veeam ONE Monitor and
• Veeam ONE Reporter.

Marie Kondo’s 6 Basic Rules of Tidying (Applied to Infrastructure)

1. Commit yourself to tidying up

You might have inherited an environment or been through years of changes that have taken the toll on your infra hygiene. The first step is telling yourself “Enough is enough! I will make things better!”. When we see a mess, its quite easy to feel overwhelmed. When we face physical clutter, the clutter bombards our senses. This could be visual, tactile – even smell sometimes (ew!). In our infrastructure environments, this can be tweaked a bit as follows:

  • WHERE DO I START?????
  • Multiple parties involved.
  • Time required to perform changes.
  • Is there an as-built document or any form of documentation highlighting the purpose of existing components.

There are probably a lot more you ask yourself but these are my FAQs followed usually by “OH MY GOD! WHYYY?” sprinkled through the conversation. In order to commit to tidying up, the first thing is setting up some time for yourself and it won’t even be 5 minutes to start with. We need to know what we are walking in to first.

To do this, I will build a special report. You could use the Inventory report (VeeamONE monitor> Reports>Configuration>Inventory) but I’m building a report with info I particularly want to hone in on. i.e. to only see what virtual machines I have and their IP Address. The idea is to reduce that bombardment of senses. My report will be built on VeeamOne Reporter by accessing WorkSpace>Custom Reports> Custom Infrastructure.

As you can see in the above image the 4th entry stands out, I have this machine with no IP, no DNS entry taking up 40GB. So I can launch an investigation and identify what this machine does.

The purpose for my criteria:
• Computer Name: I match this with the IP Address and identify if this machine is valid. For example, if I have a certain range for prod machines and another for test machines, this will stand out.
• Name: I compare this to the DNS and IP allocation. As you can see, my concern was immediately drawn to #4.
• GuestOS: I prefer my machines to be consistent but , some in house apps are built and need to be on a dedicated OS that is supported. This will help me outline my document.
• Network: Our environment only has one network but if I had preferred networks, this helps things stand out.
• Power State: I can start questioning why this machine is offline and if they’ve been offline for a while, is there a reason?
• Version: Gotta keep those versions up to date!

There are other things I can have such as VMware tools, etc which you most certainly can add but this is again limited based on my criteria of not overloading and over stressing my senses.

2. Imagine your ideal lifestyle

This is meant to be about thinking what ind of house/lifestyle you want to live in. Tweak this to a hygienic environment. No one likes an environment that throws up warnings and sadness! You might get silly now and say “I never want to work!”. Automation is what you are after (I gotchu fam!). After years of neglect, your hosts and VMs are surely going to take a toll. So let’s take a look at our environment and fix that stuff up!

The first thing to do on this, is to go to Veeam ONE monitor and take a look at what is happening right now and fix the surface issues. If there is a deeper issue, this is how we find out.

Eesh!!! Let’s fix a few things up. Now I won’t take you through all the fixes I shall make but let us focus on the 2nd error “Orphaned VM backup snapshot” to show you the automation part. If I click on the hyperlink of that machine, it will take me to a neat screen that doesn’t just tell me I have a problem but it gives me a solution AND I can authorise it to clean up!

But I’m not done here. Remember that list we made earlier? Perhaps we’ve noticed there are a bunch of machines we need to get rid of. I wonder what savings I will have. Maybe I’ve realised with this clutter, I want a management tool which requires a VM to be installed. I mean, looking at the memory usage in the earlier image, can my host handle this? Let’s find out!

We’ll go to VeeamONE Reporter again, Click the tab “Workspace” and go to “All Deployment Projects”.

In this section, I can add a scenario to either add or evict (YEET!) machines out. My criteria is as per below:

Now the initial actually passed because these are workable loads so I created another report that is over exaggerated. It presents me with a 3 page report. First page is an overview, Second page is my compute resources and Third page is recommendations.

Page 1

Page 2

Page 3

3. Finish discarding first

It is nice to have these reports and monitors but don’t just box up and put the stuff away in storage. We need to get rid of the machines you’ve identified (and confirmed with other teams) is not required. Using the above Deployment Project, being a PDF you can use this as a supporting document to sign off on the eviction/addition of VMs.

But what about all the other little crumbs and specks of dust left behind? On VeeamOne Monitor, you will have access to my favourite report called “Garbage Files”. To access this, on Veeam ONE monitor, click Reports> Datastore> Garbage Files. This report shows me consumption of files that do not belong to a VM s in my inventory. 90.35GB! Another good report in the same section is the Idle VMs. What this does is it identifies CPU, Memory, Disk Usage and Network under a certain rate.  If you are worried about Over provisioned VMs, take a look at the report “Oversized VMs”.

Idle VMs

Oversized VM report

4. Tidy by category, not by location

Perhaps you are indeed dealing with an environment that has multiple shareholders. In this case, you will want to schedule and stagger your approach instead of hitting the big red delete button. So we can plan this by first categorizing VMs. To do this, we use the Business View feature on Veeam ONE Monitor. You can categorise your VMs and provide this report to those stake holders. Till approval is received, you can monitor those VMs in the category and let that team member who protests that the VM  labelled TEST_10345 which hasn’t been powered on for 3 years is needed and frequently used, you can tell them – Nope. In fact, get the department head to do that by creating a dashboard and sharing only those categorised VMs using Veeam ONE Reporter! The below was based of existing widgets and targeting 2 machines. If the user clicks on the three lines, they can view the entire report.

Dashboard View for external users

5. Follow the Right Order

So, Marie Kondo states to follow an order. Clothes, Books, Papers, Misc. and finally sentimental items. The order follows by starting on what is the easiest. My way is not official like Marie Kondo but this is how I’d reclass it:

  • Clothes = VM Removal:  Once I’ve reviewed my VMs and know which ones to get rid of that do NOT require additional stakeholders, I’ll start on the ones my clean up. Maybe even rename and work on those machines that are not following the nomenclature.
  • Books = Resizing : Resizing the environments that could be oversized/underprovisioned. I can also consider running the VM Configuration Assessment to see where I might have potential issues with my backup solution.
  • Paper = Idle/Garbage files: Cleaning up remnant files identified by the garbage files report and removal of data from the “clothes” point.
  • Misc = Removing VMs involving other shareholders: VMs cannot be attributed to a purpose. For example, a VM named TEST_10345 and people insisting it is needed but you can identify this is an idle VM. They are not categorised, they are not in your ability to control 100%. This may take a while and is second hardest.
  • Sentimental Items= Alarm cleanup: Now this is meant to be sentimental but you may not share the same emotional sentiments with your personal life. Reviewing existing alarms and resolving the easy ones via automated actions might seem easy, such as a orphan snapshots that have remediation built in. Treat these as your first napkin on your first date at a restaurant. But removing memorabilia such as a corrupted video of your first born’s first steps is a very difficult decision. Even if you sought to fix the tape SOME day. Alarms such as a storage connection failure or issues that involve other teams might be difficult to instantly resolve. These alarms need to be handled in a structured manner.

You may have a different order and it definitely prioritize other things over the other. The important thing is leave the hardest for last.

6. Ask yourself if it ‘sparks joy’ are you happy with your progress

Marie Kondo uses the phrase to “ask yourself if it sparks joy”. You can’t really do that with infrastructure because I can’t think of a scenario a VM in particular can give you joy. If anything, when you delete an unwanted VM that was consuming massive amounts of RAM and/or CPU might spark joy when you see the host CPU/RAM warning drop back to a green from a red.

However, ask yourself if the progress you make in each step or even each order (clothes, books, paper, etc) is progress to your achieved goal. Cleaning an infra is hard work and you need to keep yourself motivated. In messes whether they are your infra or personal life, we often give up mid way when we feel the pressures of life take on. Please use the below image that motivates me and reminds you of how awesome you are and the good work you do:

 

Happy Tidying!