Re: Management command failed in KVM for SR-IOV
I have a few more computers that work on older 20Gbs devices, the new ContectX-3 and server I bought to test VM functionality. As you suggested I moved OpenSM to the only one new dev I have at the...
View ArticleClik here to view.
Can I use FDR and QDR on the same infiniband switch?
I want to make a small cluster using the upcoming SX6012 switch with a FDR ConnectX-3 adapter for the main server and QDR ConnectX-2 adapters for the nodes. From my readings into infiniband this...
View ArticleRe: Management command failed in KVM for SR-IOV
Hi, Still nothing... hope this info can be helpful. I notice that OpenSM must be started on hypervisor host in my case this is S1 otherwise the virtual function's ports are linked up but have state...
View ArticleRe: ESX 5.1.0 IPoIB NFS datastore freezes
memo (for myself) Modifications to the environment during my tests: Infiniband card was exchanged to an ConnectX PCIe gen2 (MT26418). The newer chip with PCIe 5.0GT but still not an ConnectX2 card....
View ArticleRe: Can I use FDR and QDR on the same infiniband switch?
Shouldn't be a problem. the Infiniband spec defines backward comparability. What are you planning on running between those nodes?
View ArticleClik here to view.
Re: ESX 5.1.0 IPoIB NFS datastore freezes
Hm, seems as the error comes from using one port of an infiniband card as 1) VMKernel NFS Interface2) and Network interface for the VM See picture below.After separating the VM network and the Vmkernel...
View ArticleRe: MHGH28-XTC firmware 2.9.1000 non-functional with ESXi 5.1?
Better late than never. We faced similar issues especially with a X58 chipset. The regressionoccured between firmware 2.7.9112 and 2.7.8100. More in the following thread:...
View ArticleRe: Can I use FDR and QDR on the same infiniband switch?
Do you mean the specs for the nodes, what software I will be using, or something completely different? If you mean specs the nodes are going to be 5 Supermicro 1U Twin 1026TT-1BQF servers for a total...
View ArticleClik here to view.
Re: MHGH28-XTC firmware 2.9.1000 non-functional with ESXi 5.1?
Markus Stockhausen - Cool, interesting info in that Bugzilla thread. Especially about the intel_mmu command line parameter for EL6.
View ArticleRe: Proper Configuration for IB-FDR and RoCE
Hello Michael, From what I see the configuration you are describing should be workable. When you described your issue you said that the connections became confused and stalled at times. I feel you...
View ArticleRe: opensmd fails to start on connectx-3 card
Hi, note that opensm can be run as a daemon -- are you sure there are no other instances running on the system? If no, then you may be missing some of the underlying userspace libraries needed for the...
View ArticleRe: opensmd fails to start on connectx-3 card
05:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3] Subsystem: Mellanox Technologies Device 0049 both ports are in ETH mode, however changing to ib mode fails with this...
View ArticleRe: opensmd fails to start on connectx-3 card
hmmm.... I just tried the connectx_port_config command on my HCA and it worked as shown below. what version of OFED are you running? You can type... [root@localhost ~]# ofed_info | head...
View ArticleRe: opensmd fails to start on connectx-3 card
MLNX_OFED_LINUX-2.0-2.0.5 (OFED-2.0-2.0.5): the card itself is connected to an ethernet switch, for RoCE, however this shouldnt preclude IB mode should it?
View ArticleRe: opensmd fails to start on connectx-3 card
My HCA is also connected to an Ethernet switch and I was able to run the connectx_port_config command. Can you provide the output of: # mstflint -d 05:00.0 q I want to check your HCA's PSID.
View ArticleRe: opensmd fails to start on connectx-3 card
Branko,Of course, here is the output. Image type: ConnectXFW Version: 2.30.3000Rom Info: type=PXE version=3.4.142 devid=4099 proto=ETHDevice ID: 4099Description: Node Port1...
View ArticleRe: opensmd fails to start on connectx-3 card
Well, I think that's unfortunately the issue. A PSID of MT_1080120023 means you have a ConnectX-3 Ethernet NIC (OPN of MCX312A-XCB) which I believe can only be configured for Ethernet and not...
View ArticleRe: opensmd fails to start on connectx-3 card
interesting. these cards are just EN, however IB support in the VPI line is required to achieve RoCE?
View ArticleClik here to view.
Re: opensmd fails to start on connectx-3 card
You should still be able to run RoCE on your Ethernet NIC. As long as you have OFED installed, which you do, you should be good to go. Here's a link to some documents that can assist you with RoCE...
View ArticleRe: Proper Configuration for IB-FDR and RoCE
Thanks, that's a good point.... I guess the OpenMPI stack things that ithas 2 phy transports and trying to load-share, runs into connectivityproblems. Do you think we can have both RoCE and IB active...
View Article