RDMA vs TCP/IP = latency war ...
You must check latency in test with another tool with multi-user configurations.
I'm also test it.
By increase client count, there are huge deffrence in latency factor on same throughput usage.
BR,
Jae-Hoon Choi
↧
Re: iSER for ESXi 6.5 No target detected, no traffic sent.
↧
Re: PFC on esxi 6.5
Hi Grant,
Please see the following user manual Mellanox ConnectX-4/ConnectX-5 NATIVE ESXi Driver for VMware vSphere 6.5.
Please see section 3.1.4 Priority Flow Control (PFC) :
Thanks,
Karen.
↧
↧
Re: PFC on esxi 6.5
Karen, Thank you for the reply.
Can you tell me what is Mellanox's recommended configuration if there is just 1 traffic flow? My connectx-4 cards are direct connected between two servers and used only to connect the iser initator to the iser target. There is no other traffic on these nics. I have seen references to creating a vlan and assigning it to one of the priority queues yet none of this applies to my scenario. Should i just run global pause?
↧
Re: 40G SR4 NIC Support
You mean you connect QSFP to a CX4 card ?
↧
Re: 40G SR4 NIC Support
No, the 40GBaseSR4 QSFP has an MPO12 fibre connector which is then split into 4 independent streams (LC fibre) coming from different devices which use 10GBaseR. I would like to receive data from all streams (transmit is not important).
↧
↧
Re: Melanox grid director 4036e won't boot.
No unfortunately that does not work.
The run flash_self_safe forces the switch to boot from secondary and we get the exact same error output.
From uboot I attempted to download an image by tftp but the file transfer begins, the switch outputs an error and boots to the same error.
I opened the unit and there are 4 red LEDs so I suspect a hardware failure.
The LEDS are as follows:
Top row of LEDS (next to RAM module, below the chassis fans)
D104 + D105 on RED
LEDS in bottom right of chassis
CPLD 2 R643 - D87 RED
CPLD 4 R645 - D89 RED
The LEDs turn red shortly after power is applied.
Do you know what may have failed? Will the failed components be replaceable as this is a legacy unit and this with another 7 switches may have a similar problem.
Kind regards.
Rav.
↧
Re: Melanox grid director 4036e won't boot.
In this case, I think you need RMA the switch if the switch under warranty
↧
NFS over RDMA on OEL 7.4
Hello my configuration is simple OEL 7.4 two Mellanox ConnectX-3 VPI cards , SX1036 switch and two very Fast NVMe drives.
So my problem is that I configured NFS over RDMA using Infiniband Support packages from OEL because OFED from Mellanox not support
NFS over RDMA from version 3.4 + .
Everything is working I can connect to server over RDMA and I can write/read from NFS server etc. but I have a problem with performance .
I done test on my stipe LV and fit shows me 900k IOPS and around 3,8 GB/s using 4k but when I do the same tests on NFS client I can't get more
then 190k IOPS ? Problem is not the bandwidth because when I change the block size I can get even over 4GB/s but the problem seams to be number
of IOPS delivered from server to client.
I am asking maybe somebody have idea ?? I already change size and size to 1m but without any performance benefits.
My next steps will be configure Aggregation (LCAP) to see if it change something , now I'm using only one Port .
Adam
↧
Re: Melanox grid director 4036e won't boot.
Legacy equipment out of warranty.
Are we able to purchase a service contract or is this equipment unsupported?
Many thanks.
↧
↧
Re: Melanox grid director 4036e won't boot.
You should contact sales guys
↧
Re: Melanox grid director 4036e won't boot.
Thank you for the assistance.
This call can now be closed.
↧
Re: Running ASAP2
Hi Francois,
Please check https://community.mellanox.com/docs/DOC-2964
and unmerged yet patchset https://review.openstack.org/#/c/504911/
↧
Re: Melanox grid director 4036e won't boot.
Hello Rav,
Mellanox do not seel servicecontracts on 4036E switches anymore. The product is at EOL stage.
for more information, please refer to our EOL info page at: http://www.mellanox.com/page/eol
Sorry we couldn't assist you.
Thanks
↧
↧
Re: Running ASAP2
Hi Lenny
I can’t access this document.
Can you please send me the PDF version.
There seems to be something wrong with my Mellanox account.
Regards
Francois Kleynhans
↧
Re: Running ASAP2
I also cannot access the documents listed...
↧
Re: How to configure MCX354A-FCBT Mellanox InfiniBand speed at 56Gbps ?
From the output prints you have presented, it looks like your SX6036 switch should be good & supports 56Gb (FDR)
- cables & nics are also fine and capable of fdr
- cables present: Infiniband speeds : SDR , DDR , QDR , FDR
- switch presents:
Supported LLR speeds : FDR10, FDR - which indicates you can set it to 56Gb
Supported speeds : sdr, ddr, qdr, fdr10 - here you see that fdr is missing
so, in my view - all you have to do is to run the following command on the switch that will add FDR on "supported speeds" list
run on CLI commnd:
(config) # interface ib <interface#> speed sdr ddr qdr fdr10 fdr force
(config) # configure write (to save the changes)
you should now see: Supported speeds : sdr, ddr, qdr, fdr10, fdr
this should enable you to uses 56Gb on the switch and on the nics as well
↧
Re: Running ASAP2
Please try now.
↧
↧
Re: Running ASAP2
Thanks I can access it now
↧
Re: NVMeOF SLES 12 SP3 : Initiator with 36 cores unable to discover/connect to target
Hi,
Adding the parameter didn't help. It still gives the same error:
athena:~ # nvme discover -t rdma -a 192.168.0.1 -s 4420
Failed to write to /dev/nvme-fabrics: Invalid argument
athena:~ # dmesg |tail -1
[ 1408.720843] nvme_fabrics: unknown parameter or missing value 'hostid=a61ecf3f-2925-49a7-9304-cea147f61ae' in ctrl creation request
athena:~ # nvme connect -t rdma --nr-io-queues=32 -a 192.168.0.1 -s 4420 -n mcx
Failed to write to /dev/nvme-fabrics: Invalid argument
athena:~ # !dm
dmesg |tail -1
[ 1437.914081] nvme_fabrics: unknown parameter or missing value 'hostid=a61ecf3f-2925-49a7-9304-cea147f61ae' in ctrl creation request
↧
Re: How to configure MCX354A-FCBT Mellanox InfiniBand speed at 56Gbps ?
HPE Part Number 712498-B21 is SX6036T FDR10 switch.
Therefore in this case user can't use FDR14 56Gb speed in fabric.
HPE Support document - HPE Support Center
Why did you give a incorrect information to community users?
Best Regard,
Jae-Hoon Choi
↧