UPDATE 8/15/2020 – I now have purchased a thunderbolt 3 to 10 Gbps ethernet adapter. With this, I can finally test actual 10 Gbps in my network.
UPDATE 7/25/2020 – I have discovered the comparison between iOS apps turned out to be Download vs. Upload stream and the throughput has been affected asymmetrically; hence, it gave me false sense of solved issue.
There are many posts/complaints about UniFi’s current generation SFP+ port throttling its throughput. I myself had been experiencing one of this issue. Today, I have successfully identified an issue and solve it. The issue/solution is quite a simple for my case but for those of you interested in troubleshooting your own network issue, this may provide a guide of how I eventually arrived to my solution though I wished I had figured this out sooner.
Setup
SFP+ port is capable of 10Gbps throughput. These ports are equipped on UniFi Dream Machine Pro and UniFi Switch Pro Second Generation PoE switches. My current setup indeed consists of these products with UDMP as my gateway and USW Pro Gen 2 24 PoE as my main switch.
SFP ports are generally used as backbone connection between gateway and switches. So this is indeed what I have.
My current internet service is cable from Xfinity with 1Gbps down and 40 Mbps up.
UDM Pro to USW Pro 24 Gen 2 are connected via SFP+ ports. USW Pro 24 Gen 2’s second SFP+ port is connected to my Qnap NAS has a 10Gbps Ethernet port. Iperf 3 serve is running on my NAS.
SFP ports require adapters. Initially, I had purchased following SFP+ adapters.
- SFP+10GBASE-T Transceiver Copper RJ45 Module by HiFiber ($42.99) x 2
- 10G SFP+ RJ45 Copper Transceiver, 10GBase-T Module by iPolex ($39.99) x 1
Cable used were Cat 6 cables. For cleaner setup and as part of troubleshooting, I have subsequently purchased Ubiquiti brand 10G DAC cable and 10G SFP+ RJ45 copper transceiver.
So the most recent setup up until today, I had UBNT brand 10G DAC between UDMP to USWP 24. USWP 24 to NAS were connected using HiFiber 10G Transceiver with Cat 6 cable attached to it.
SFP+ Port Throughput Issue?
The first thing I noticed was my iPerf test setup on NAS system was far slower than the what I expected. So I’ve done following four tests.
The issue is on the yellow row i.e. test A where iPerf down speed was only 300. I actually ad full 600 upload speed here.
The fact, I got full speed on internet speed tests, it confirms connection between switch to UDMP, and to the internet is fully functional.
Then looking at Test A and C, the only difference was the use of switch port. This lead me to conclude SFP+ path is the issue. This still left me a very strange feeling because test B shows SFP+ path actually has full speed when client and switch uses wired connection, which is downstream to SFP+ path to the NAS so very puzzling. Nonetheless, my conclusion was SFP+ path at this point because I have also seen others having throughput issue on the official forum and concluded they are related to SFP+ port.
Server -> 48 POE 500 -> UDM Pro -> Century Link ONT
My century link connection is 1gbps in each direction, and is attached to the UDM Pro via SFP copper ethernet module on the WAN port. The century link connection is vlan 201 + pppoe.
The server is connected via 10GBe optical connection to the SFP+ port on the 48 POE.
When I connect the 48 POE to the UDM Pro using 10GBe SFP+ optical, I get reduced speed test results. Around 800mbps download, and 500 mbps upload. These tests are pretty consistent.
However, when I connect the 48 POE to the UDM Pro using regular 1000baseT copper ethernet, I get ~900mbps download and ~900mbps upload, which is what I expect to get from my century link connection.
https://community.ui.com/questions/Strange-WAN-Bandwidth-Issue-with-UDM-Pro-and-48-POE-500/02e5dbbb-b93f-405f-a576-0caf0553e343#answer/003438e8-ab95-42e9-9816-36aedc1e8f6d
Replacing SFP+ Transceiver Unit
According to UniFi, neither of SFP+ module I purchased are listed under officially supported 10G module despite HiFiber specifically says compatible with Ubiquiti. So I have decided to purchase the official Ubiquiti branded transceiver unit.
However, after replacing SFP+ transceiver module the issue still persisted. Since I am now using Ubiquiti’s official branded transceiver I know it is no longer the compatibility issue of the module. This is finally when I realized I should have tested another scenario with iPerf.
The real issue?
The key here is test G. I’ve now added MBP wireless iPerf test. It is actually reaching the full speed. The issue is still rather strange but now comparing Test E and G, the difference becomes client.
Since Test C as well as internet speed tests have proven that hardware itself is not an issue, the next troubleshoot target is an actual app.
iOS iPerf App
I have been using iPerf 3 Wifi Speed Test app which is the highest rated iPerf app on App Store at the time of this writing, and very simple to use; however, this app does not allow command line arguments. In fact, for me to get the peak throughput number on MBP, I need to specific iPerf command -P (# of stream) argument. Even though this app allowed stream number from 1 to 5 and certainly was working with while I had ethernet connection between switch and NAS, this is the only remaining piece I could reasonably attribute the potential source of the issue.
So I have also tried another app, HE.Net app. This also gave the same lower throughput speed. In fact, this is something I have done very early stage but because two Apps showed same numbers, I assumed App was not an issue initially.
But this time, I had no other explanation since I am now using UBNT branded SFP+ module. So I’ve purchased iPerf Speed Test Tool ($1.99) as it takes command arguments. With this app, I use following command (as I do on my MBP).
iperf3 -P 5 -c 192.168.1.9
Now I have actually got full speed on my iPhone 11 Pro with iPerf test.
Upload vs. Download
Added 7/24/2020
Unfortunately, I was wrong yet again. Using debug mode of iPerf server side, I have discovered that iPerf 3 Wifi Speed Test uses -R argument. This argument reverses data transfer.
Reverse test mode – Server sends, client receives
https://iperf.fr/iperf-doc.php
Now knowing that what was happening on my test above were simply comparing download stream to upload stream. So I have re-run whole test in both directions and here is the result.
Now it is obvious the issue is very specific. Basically, when SFP+ port is utilized iPerf Down speed on my WiFi client reduces to 300’s independent of transceiver module or client type.
Another issue
The next day after I conducted original tests with UBNT brand SFP+ module, all speed tests including wired MBP were significantly reduced whether it was upload or download. This had spontaneously resolved later the same day. As far as I can tell, I did not have any specific network load during the time so not sure what was happening there.
10G Network Test
Added 8/15/2020
I have purchased this new adapter and I have had my old Netgear switch with multi-gig speed capability including one SFP+ 10Gbps port and one RJ45 port. So with these components together, I can now test one full end to end 10Gbps pathway.
The setup here is basically Qnap NAS running iPerf 3 server accessed by MacBook Pro 15” 2016 model. For the link between USW Pro 24 Gen 2 and Netgear switch, I have used UniFi DAC. SFP+ adapter on USW Pro 24 Gen 2 also used UniFi adapter.
Issue and Solution
In conclusion, the issue had always been “iOS client App for iPerf testing”. But what made this really difficult was it is conditional. The app acts up only when SFP+ port is utilized.
To be honest, I don’t know why this happens. It must be is somehow behind the seen iPerf3 WiFi Speed test app uses different command, perhaps additional argument that only affects when SFP+ port/10G connection is used.
So the key learning points on this experience for me was, I should have stuck with my typical systematic approach but should have been complete. If I had done wireless MBP test in first place, I might have saved $65 (plus tax and shipping) as purchasing UBNT branded transceiver did not really help in this case.
The solution remains unclear. However, the issue is definitely on the SFP+ port. It is asymmetric throughput reduction of download (iPerf with -R argument) while client traffic originates from Wi-Fi but not with wired client nor upload traffic.