My homelab originally used a Cisco Catalyst 3560X as the central switch between the Proxmox host, ESXi host, physical Windows client, and another physical system.

The switch was useful for learning enterprise switching, VLAN configuration, access ports, trunks, and Cisco IOS. However, there was one disadvantage to keeping an older enterprise switch running continuously: power consumption. Some calculations from another chat pro Jahr = per year.

Gerät	Angenommene Nutzung	Verbrauch pro Jahr	Kosten pro Jahr
Cisco WS-C3560X-24T-S	93 W, 24/7	814,7 kWh	285,14 €

There are other measures I have to take, but this was the easy first step in reducing costs.

After reviewing the electricity consumption of the homelab, I decided that the Cisco switch no longer needed to be part of the permanent 24/7 infrastructure.

The goal was therefore not to remove the Cisco from the lab completely. Instead, I wanted to change its role:

The Cisco switch remains available for networking labs, but is powered off during normal homelab operation.

The challenge was to remove the switch without buying another switch and without redesigning the entire network.

Fortunately, the existing Proxmox and ESXi network configuration already provided almost everything required.

Existing network design

The relevant physical systems were:

System	Role	Network
ATHENA	Proxmox host / OPNsense	192.168.10.0/24 and home network
ARES	VMware ESXi host	192.168.10.0/24 and LAB30
AETHER	Physical system	192.168.10.0/24
STEUERN	Physical Windows client	192.168.30.0/24
Cisco Catalyst 3560X	Core/access switch	VLAN 10 and VLAN 30

The Cisco configuration connected the systems approximately like this:

Gi0/1   Access VLAN 10   -> ARES vmnic0
Gi0/2   Access VLAN 10   -> AETHER
Gi0/3   Access VLAN 30   -> STEUERN
Gi0/23  Access VLAN 30   -> ARES vmnic1
Gi0/24  Access VLAN 10   -> ATHENA ETH1

This meant the Cisco switch was providing two separate Layer-2 networks.

The management/server network used:

192.168.10.0/24

The Windows lab network used:

192.168.30.0/24

The important question was whether the existing hypervisor hosts could provide the required Layer-2 connectivity without the Cisco.

Checking the Proxmox network configuration on ATHENA

The first important discovery was already visible under:

ATHENA → System → Network

ATHENA's existing Proxmox network configuration. vmbr10 already contains three physical Ethernet interfaces and can therefore provide Layer-2 switching between physical systems without an external switch.

The relevant configuration was:

enp1s0
└── vmbr0
    ├── 192.168.178.50/24
    └── Gateway 192.168.178.1

This is the connection to the normal home network and FritzBox.

The more interesting configuration was:

vmbr10
├── enp2s0
├── enp3s0
├── enp4s0
└── 192.168.10.2/24

This meant that three physical Ethernet interfaces were already connected to the same Linux bridge.

A Linux bridge in Proxmox behaves like a software Ethernet switch. Traffic entering one bridge port can be forwarded to another bridge port without being routed.

CLI^of the important network settings in /etc/network/interfaces:

In other words, ATHENA was already capable of doing this:

             vmbr10
                |
      +---------+---------+
      |         |         |
   enp2s0    enp3s0    enp4s0
      |         |         |
    Device    Device     Device

No additional Proxmox bridge had to be created.

Identifying ATHENA's physical interfaces

This is a diagram that I made for myself in order to understand all that terminology flying around inside Proxmox and what these connections actually refer to. Here is how Proxmox "Athena", my Ubuntu "Test" and the Topton N100 Mini Server interfaces work together:

The existing documentation also showed the physical interface mapping:

ETH0 = enp1s0
ETH1 = enp2s0

ETH0 is connected to the FritzBox side of the network:

ATHENA ETH0 / enp1s0
        |
      vmbr0
        |
192.168.178.0/24
        |
     FritzBox

ETH1 previously connected ATHENA to the Cisco switch:

ATHENA ETH1 / enp2s0
        |
      Cisco

Because enp2s0, enp3s0, and enp4s0 are all members of vmbr10, the Cisco could be replaced by direct connections to ATHENA. Here is an old photo of the hardware we are working with.

Existing VLAN 30 configuration on ATHENA

ATHENA also contained:

enp2s0.30
└── vmbr30

This was created for earlier VLAN labs.

However, the later Windows lab had become tied to the physical Cisco infrastructure instead. The active LAB30 network was now handled by ARES/ESXi and the Cisco access ports.

The old vmbr30 configuration was therefore not required for the new permanent topology.

I deliberately left it untouched during the migration.

This follows an important administration principle:

Do not remove unrelated configuration while performing a network migration.

Once the new topology has been stable for some time, the old Proxmox VLAN30 configuration can be removed separately as a cleanup task.

Checking the ESXi port groups

The next step was to inspect the ESXi network configuration on ARES.

Under:

Netzwerk → Portgruppen

the host contained:

LAB-VLAN30
VM Network
Management Network

ESXi port groups before removing the Cisco switch. LAB-VLAN30 is connected to vSwitch-LAB30, while the normal management network uses vSwitch0.

The important configuration was:

LAB-VLAN30
VLAN-ID: 0
vSwitch: vSwitch-LAB30

The management network also used VLAN ID 0.

In ESXi, VLAN ID 0 means that ESXi itself is not adding an 802.1Q VLAN tag to the Ethernet frames.

That was important because the Cisco ports used for these systems had been configured as access ports.

From the endpoint's perspective, the traffic was therefore already ordinary untagged Ethernet.

Verifying the physical ESXi uplinks

ARES contains two physical Intel network interfaces:

vmnic0
MAC: b4:96:91:51:26:bc

vmnic1
MAC: b4:96:91:51:26:bd

Both were operating at:

1000 Mbit/s
Full Duplex

The detailed vSwitch-LAB30 view confirmed:

vSwitch-LAB30
└── vmnic1

vSwitch-LAB30 uses physical adapter vmnic1. The LAB30 port group uses VLAN ID 0, which allows the physical Windows client to be connected directly to this adapter without configuring VLAN tagging on the client.

The virtual machines connected to this network included:

DC01
CLIENT01
WSUS01

Their relevant lab addresses are:

DC01       192.168.30.1
WSUS01     192.168.30.20
CLIENT01   192.168.30.50
STEUERN    192.168.30.51

This created an interesting possibility.

Instead of:

ARES vmnic1
     |
Cisco VLAN30
     |
STEUERN

the two systems could simply be connected directly:

ARES vmnic1
     |
     |
STEUERN

The ESXi standard switch would continue to connect STEUERN with the LAB30 virtual machines.

Final switchless topology

The resulting physical topology is significantly simpler.

                         FritzBox
                            |
                            |
                    ATHENA / Proxmox
                            |
             +--------------+--------------+
             |                             |
          vmbr10                         vmbr10
             |                             |
          ARES                           AETHER
        vmnic0
             |
        ESXi vSwitch0


          ARES
        vmnic1
             |
             |
          STEUERN
             |
      192.168.30.0/24

The permanent connections are now approximately:

ATHENA enp1s0 / ETH0
        |
     FritzBox

ATHENA enp2s0 / ETH1
        |
   ARES vmnic0

ATHENA enp3s0 or enp4s0
        |
      AETHER

and:

ARES vmnic1
        |
      STEUERN

One of ATHENA's vmbr10 physical ports remains available for future use.

Replacing the VLAN 10 Cisco connections

Previously the VLAN 10 path looked like this:

             +------ ARES
             |
ATHENA ------ Cisco
             |
             +------ AETHER

After the migration:

             +------ ARES
             |
ATHENA ------+
             |
             +------ AETHER

ATHENA's existing vmbr10 Linux bridge now provides the Layer-2 connectivity previously provided by the Cisco access ports.

The IP configuration did not need to change.

Relevant addresses remain:

OPNsense        192.168.10.1
ATHENA          192.168.10.2
LibreNMS        192.168.10.5
Wazuh01         192.168.10.20
ARES ESXi       192.168.10.50
ARES VM         192.168.10.100
AETHER          192.168.10.101
DC01 OUT        192.168.10.103
Rocky           192.168.10.113
Test            192.168.10.123

The virtual machines on ATHENA were unaffected because they were already connected internally to vmbr10.

Replacing the VLAN 30 Cisco connections

The previous LAB30 topology was:

DC01 / CLIENT01 / WSUS01
          |
    vSwitch-LAB30
          |
       vmnic1
          |
    Cisco Gi0/23
          |
       VLAN30
          |
     Cisco Gi0/3
          |
       STEUERN

The Cisco was simply providing Layer-2 forwarding between two access ports.

The new design removes it:

DC01 / CLIENT01 / WSUS01
          |
    vSwitch-LAB30
          |
       vmnic1
          |
       STEUERN

No Windows VLAN configuration was necessary because the LAB-VLAN30 ESXi port group already uses VLAN ID 0.

STEUERN therefore continues to behave like an ordinary Ethernet client.

VLAN names versus actual VLAN tagging

There is one useful distinction in the new design.

I still refer to the networks as:

VLAN10
LAB30 / VLAN30

because these names describe their role in the homelab and preserve the existing documentation.

However, after removing the Cisco, the permanent physical links are no longer dependent on 802.1Q VLAN tagging.

For example:

ARES vmnic1 <-> STEUERN

carries untagged Ethernet.

The network is still:

192.168.30.0/24

but the direct cable does not need a VLAN tag because there is only one Layer-2 network on that link.

The same principle applies to the direct VLAN10 connections through ATHENA's Linux bridge.

The VLAN terminology therefore describes the logical lab network, while the permanent direct links themselves are now simpler untagged Ethernet segments.

Testing the new topology

After moving the physical cables, the Cisco switch was no longer part of the traffic path.

The first successful test was AETHER.

AETHER retained connectivity through its new direct connection to ATHENA's vmbr10.

The ESXi management interface was then tested at:

https://192.168.10.50

The ESXi Host Client remained reachable.

This confirmed the new path:

AETHER
   |
ATHENA vmbr10
   |
ARES vmnic0
   |
ESXi

The LAB30 connection was then tested from STEUERN.

STEUERN successfully reached:

192.168.30.1

which is DC01's LAB30 interface.

This confirmed:

STEUERN
   |
ARES vmnic1
   |
vSwitch-LAB30
   |
DC01

Additional domain and network connectivity also continued to work.

At this point all relevant systems were operational without the Cisco switch.

Result

The Cisco Catalyst is no longer required for normal homelab operation.

The permanent infrastructure now uses capabilities that were already available in the existing hardware:

Proxmox vmbr10 provides Layer-2 connectivity for the physical 192.168.10.0/24 systems.
ESXi vSwitch-LAB30 provides Layer-2 connectivity for the 192.168.30.0/24 Windows lab.
ARES vmnic1 connects STEUERN directly to the virtual Windows lab.
No additional switch was purchased.
Existing IP addresses remained unchanged.
Existing Windows domain, Intune hybrid, WSUS, and other lab configurations were preserved.

Most importantly, the Cisco can now remain powered off during normal operation.

It is still available whenever I want to work on:

Cisco IOS
VLANs
802.1Q trunks
STP
switch security
ACLs
enterprise switching

This is a better role for an older enterprise switch in a small homelab than keeping it powered continuously simply to connect a handful of systems.

Final topology summary

HOME NETWORK
============

FritzBox
192.168.178.1
    |
    |
ATHENA enp1s0
vmbr0
192.168.178.50


MANAGEMENT / SERVER NETWORK
===========================

                    OPNsense
                  192.168.10.1
                        |
                     vmbr10
                        |
        +---------------+---------------+
        |                               |
      ARES                            AETHER
    vmnic0                        192.168.10.101
192.168.10.50

ATHENA and its VMs also remain connected to vmbr10.


WINDOWS LAB NETWORK
===================

DC01          192.168.30.1
CLIENT01      192.168.30.50
WSUS01        192.168.30.20
        \        |        /
         \       |       /
          vSwitch-LAB30
                |
              vmnic1
                |
             STEUERN
          192.168.30.51


CISCO CATALYST
==============

Powered off during normal operation.

Powered on only when required for networking labs.

TO DO:

This gives me the opportunity to make an updated network diagram....

and practicing network discovery in Linux CLI

Search This Blog

Daniel Thiele IT - daniel@dproject.de

Removing the Cisco Core Switch from Daily Homelab Operation and Increasing Cost Efficiency