wanted a free solution

ends up buying a domain

Welcome to the club, buddy!

I’m probably too new to be offering advice, so I’ll just pile on the encouragement. You can do it!

That’s what I need the most. This project could be a bit ambitious for me, but I’ll go slow. Thanks a lot!

I think you have a typo here, eth1 is WAN not WLAN

Corrected!

When the VM goes down you can still configure the network manually on your PC and access proxmox directly

Yes, that’s why having this MGMT port is so important. Time to make some more cables :P

This is why the hardcoded ip will still work.

For some of the critical infra, I will probably hardcode the IP

I saw that you added the detail that you’re also running another vm - AFAIR you could route that via the management port as well with via proxmox without added complexity.

I’ve elaborated upon this in this comment in this thread. Please check and let me know your opinions.

I have one more doubt.

(Nothing setup as of now, just checking all possible angles)

Since, as per your suggestion, I have passed the NIC to OPNsense, so Proxmox won’t see the dual NIC card. The onboard ethernet port eth3 is connected to a switch and Proxmox will use eth3.

Here’s a fresh diagram

You can see that I have a Ubuntu VM. How will the PC communicate with the VM? (Links E and D are hypothetical. I don’t know which one is better given my scenario). Will it be C -> B -> D or C -> A -> E?

Case #1: C -> B -> D

The switch must differentiate between uplink traffic/proxmox traffic/other physical devices connected to the switch (phone/TV). I prefer this solution because even if OPNsense is down, I can still access the Ubuntu server. But the switch must handle this complexity. I will be getting a managed switch anyway.

Case #2: C -> A -> E

The interface eth2, which OPNsense is using, must have some sort of routing table which redirects to links E or F depending upon destination IP (Probably implemented using virtual bridge or virtual switch). This is simpler for the switch but if OPNsense is down, i lose direct network access to the VM. I can probably access it via Proxmox web-interface.

Can you tell me which one is preferred or which one you would have done?

Sorry for my ramblings.

I have saved your comment and will try to implement them when setting it up. Thank you!

Have done something similar, you’ll discover the joy of have no internet while you do server maintenance, upgrades or disaster recovery and worst case, hardware failures.

I may purchase another lower powered device to prevent a single point of failure, but I want to get this thing working. Sorry, too excited with Proxmox now :D Also, in the worst case scenario, I’ll just use my current router while I manage my OPNsense

I would enable VLANs for network segments but if you either don’t have switches that support tags or are new to all of this it’s going to be WAY less headache to just use it all untagged for now. (Spend time on funner things, seriously!)

Setting up VLANs will be my next project :D . Right now, my goal is to get this thing working.

I have messed up my timings a bit, so the dual NIC card is on the way from Amazon. I’ll post my results up here once it arrives.

Thanks, i may go this route.

FYI, you might occasionally run into issues where the NIC “GUID” changes so your VM won’t be able to start.

I think this is the same issue as a Linux host forgetting where to mount a disk since the UUID was not written in fstab.

But why does the GUID change? Can’t it be hard-coded?

Want setting change that requires a reboot from proxmox would result in a total lots of the network

Did you mean a setting change in proxmox? If yes, then I understand the risks.

Also, after the reboot does the setup comeback online automatically? Or do you need to perform some manual intervention?

So if you want to go that route I recommend you to use eth3 as dedicated port for proxmox.

I need a clarification here.

• eth1 = WAN. So it gets a public IP.
• eth2 = LAN. So it will get a 192.168.1.1 IP (this is usually hard-coded during OPNsense setup)
• eth3 = MGMT.
  • If this is out-of-band, that is, it will work even when OPNsense VM is down, how will it get the IP?
  • Should the IP for MGMT be hard-coded? Should it be in the same subnet or needs to be different?
  • If I want to access via MGMT, what will the routing table of the device on the other end (the desktop from which I will access Proxmox via MGMT) look like?

If you have any resources regarding this setup, please share.

Thank you for your response.

Edit: Changed WLAN to WAN

Are you hosting more VMs on Proxmox that need network via a virtual switch?

Only one more VM which will host some services (to be accessed via a reverse proxy)

Are you providing network to other physical devices as well via a switch, so you need to output to that?

The Proxmox is connected to a switch, and all my other devices (other servers/PC/access point) are connected to this switch. Ideally, these services will connect to the OPNsense VM and get IP via DHCP from OPNsense.

Do you want OPNsense to be your gateway and assign IPs, or do you have a router?

OPNsense should be my public facing gateway, similar to what my current router (TP-Link) is doing. Currently, my router is connected to an ISP GPON. But it seems the GPON acts only as a fiber to ethernet converter. I use my ISP provided credentials on my ROUTER to authenticate with my ISP (via PPPoE)

As you have 3 NICs and sound like you want to use them, let’s assume you aren’t doing RoaS, but this could all be done on one (very busy) NIC.

RoaS? Router as a Service? I have no idea. I want to the 3rd port, but if you feel like this is a bit complicated, I can leave it for now, provided I don’t incur a heavy penalty for using the LAN port for traffic and Proxmox management.

And … you can do this!

Thank you, I needed that! I am just starting out with Proxmox and OPNsense and it seems a bit overwhelming. I am trying to start out small. In case everything blows up, I still have my old TP-Link router.

PS: Please check the original post as I have added a diagram regarding what I want to do.