============================
R E A D M E    N O T E S
============================
QLogic qfle3 VMware ESX Native Driver for ESXi 6.7
Copyright (c) 2018 QLogic Corporation
All rights reserved
Table of Contents
=================
1.  Introduction
2.  Using the Driver
3.  Driver Parameters
4.  Feature support
5.  MultiQueue/NetQueue
6.  Statistics
7.  Features under Development.
8.  Contacting Support

1.   Introduction
     =============
This file describes the qfle3 Native VMware ESXi driver for QLogic BCM57712/57712_MF/57800/57800_MF/57810/
57810_MF/57840/57840_MF 10Gb PCIE Ethernet Network Controllers.

qfle3 is a native driver intended to replace bnx2x.  

OS Support:
  
   * Driver Version for ESXi 6.5/6.7:- 1.0.84.0 


2.   Using the Driver
     =================

Installing the Driver:
---------------------
You can use the driver zip bundle .zip or .vib to update existing driver or install new driver. Unzip the driver installer zip bundle and extract .vib file from it.
   
   The .vib or .zip file may be installed directly on an ESX server from the command line, or through the VMware Update Manager (VUM).

COMMAND LINE INSTALLATION
-------------------------

New Installation
----------------

   For new installs, you should perform the following steps:
   1. Copy the VIB or ZIP to the ESX server.  Technically, you can place the file anywhere that is accessible to the ESX console shell, but for these instructions, we'll assume the location is in '/tmp'.

   Here's an example of using the Linux 'scp' utility to copy the file
   from a local system to an ESX server located at 10.10.10.10:

#scp qfle3-1.0.XX.0-1OEM.670.0.0.8169922.x86_64.vib root@10.10.10.10:/tmp

2. Issue the following command (full path to the VIB must be specified):
   #esxcli software vib install -v {VIBFILE}

   In the example above, this would be:

   #esxcli software vib install -v /tmp/qfle3-1.0.XX.0-1OEM.670.0.0.8169922.x86_64.vib

In case you are installing using zip:
#esxcli software vib install -d /tmp/qfle3-1.0.XX.0-1OEM.670.0.0.8169922.x86_64.zip

Note: Depending on the certificate used to sign the VIB, you may need to change the host acceptance level.  To do this, use the following command:

   #esxcli software acceptance set --level=<level>

   Also, depending on the type of VIB being installed, you may have to put ESX into maintenance mode.  This can be done through the VI Client, or by adding the '--maintenance-mode' option to the above esxcli command.

Upgrade Installation
--------------------

The upgrade process is similar to a new install, except the command that should be issued is following:

   #esxcli software vib update -v /tmp/qfle3-1.0.XX.0-1OEM.670.0.0.8169922.x86_64.vib
Or

#esxcli software vib update -d /tmp/qfle3-1.0.XX.0-1OEM.670.0.0.8169922.x86_64.zip

VUM INSTALLATION
----------------

   The VMware Update Manager (VUM) is a plugin for the Virtual Center Server (vCenter Server).  You can use the VUM UI to install a VIB by importing the associated offline bundle package (a ZIP file that contains the VIB and metadata).  You can then create an add-on baseline and remediate the host(s) with this baseline. Please refer the vCenter Server documentation for more details on VUM.

Removing the Driver:
--------------------

   Removing the driver package can be accomplished using the following command:

   #esxcli software vib remove --vibname <vib-name>

   Eg. esxcli software vib remove --vibname qfle3

   To unload the driver temporarily, do the following:

   #vmkload_mod -u qfle3


3.   Driver Parameters
     ==================

The qfle3 driver settings can be queried using 

#esxcli system module parameters list -m qfle3
Or 
# esxcfg-module -i qfle3

1.	debug_mask: Enabled debug mask (default: 0)

The debug_mask module parameter should only be set and used for debug purposes as the additional logging will flood numerous messages. It is not advisable to set this parameter for regular driver usage.
 
Possible Values:
0x00000001     /* load and unload    */
0x00000002     /* interrupt handling */
0x00000004     /* slowpath handling  */
0x00000008     /* stats updates      */
0x00000010     /* packet transmit    */
0x00000020     /* packet receive     */
0x00000040     /* phy/link handling  */
0x00000080     /* not used    */
0x00000100     /* dumping mbuf info  */
0x00000200     /* register access    */
0x00000400     /* lro processing     */
0x00000800     /* uplink debug       */
0x00001000     /* qeueu debug       */
0x00002000     /* hw debug       */
0x00004000     /* cmp debug  */
0x00008000     /* start process debug  */
0x00010000     /* debug assert       */
0x00020000     /* debug poll       */
0x00040000     /* debug TXSG       */
0x00080000     /* debug crash       */
0x00100000     /* debug vlan       */
0x00200000     /* state machine      */
0x00400000     /* nvm access      */
0x00800000     /* SRIOV        */
0x01000000     /* mgmt interface   */
0x02000000     /* CNIC */
0x04000000     /* DCB */

0xFFFFFFFF     /* flying monkeys     */

2.	enable_fwdump: Enable/Disable the firmware dump file. Set to 1 to enable firmware dump, Set to 0 to disable firmware dump [Default]

3.	enable_lro: Enable/Disable the TPA (LRO) featureSet to 0 to disable TPA, Set to 1 to enable TPA [Default]

4.	hw_vlan: Enable/Disable VLAN removal/insertion by hardware. 0: Disabled and 1: Enabled. Default: 1

5.  tx_to_delay: Time interval (in seconds) to wait for before considering a Tx queue stuck and triggering a tx timeout.  Default:5, Disable:0, Minimum:5

6.	mtu: MTU when the driver is loaded. Range: 0-9000. (default: 1500)

7.	offload_flags: Offload flags: 1 (cso) 2(tso) 4(vxlan offload) 8(Geneve offload). Default: 15

8.	rssq_nr: Number of RSS Queues: 0 (Auto) or 1 to 4 (fixed queue number). Default: Auto

9.	rx_filters: Define number of RX filters per NetQueue-1: use the default number of RX filters based on availability:0: Disable use of multiple RX filters; 1,2,3,...: Will force the number of RX filters to use for NetQueue; Default: -1

10.	rxqueue_nr: Number of Rx Queues: 0 (Auto) or 1 to 8 (fixed queue number). Default: 4

11.	rxring_bd_nr: Number of RX BD Buffers: 4096(min) 16384(max), will round up to nearest power of two. Default: 4096

12.	txqueue_nr: Number of Tx Queues: 0 (Auto) or 1 to 8 (fixed queue number). Default: 4

13.	txring_bd_nr: Number of TX BD Buffers: 4096(min) 16384(max), will round up to nearest power of two. Default: 4096

14.     RSS: Number of RSS Queues: 0 (Auto) or 1 to 4 (fixed queue number). Default: 1

15.     DRSS: Number of RSS Queues associated with default queue: 0 (Disabled), 2 (min), 4 (max). Default: Disabled

16.     rss_engine_nr: Number of RSS Engines: 0 (Disabled) or 1 to 4 (fixed NUmber of RSS Engines). Default: 4  (Only applicable to EWSX6.7)

17.     enable_vxlan_filters: Enable/Disable the VXLAN Rx filters. Set to 0 to disable VXLAN Rx filters. Set to 1 to enable VXLAN Rx filters. Default: 0

18.     dropless_fc: Pause on exhausted host ring. 0: Disable (Default) 1: Enable

19.     max_vfs: Number of VFs to be enabled for each pci function. Default:0, Valid values: 0:Disable, 1 to 64 (Actual MAX VFs count dependent on Adapter Hardware)

20. fairness_threshold: When set to 1 will enable the fairness threshold; Default: 0

21. enable_dcbx: Enable DCBX. 0: Disable 1: Enable(Default)

The qfle3 driver settings can be changed using:

#esxcli system module parameters set -m qedentv -p Param=Value

Or

#esxcfg-module -s Param=Value qfle3

4.   Feature Support
     ================

Jumbo mtu
Rx/Tx Netqueue
NPAR
SRIOV

Offloads Support:-
Tx cksum for L3/L4
Rx cksum for L3/L4
TSO for IPv4 and IPv6
LRO for IPv4 
Scatter Gather IO

Note: Stateless offloads (TSO/CKO) are supported for the egress (transmit) path for VXLAN/Genève traffic. 
Adapter and driver do not support stateless offloads for VXLAN/Genève traffic in the ingress (receive) path.

5.   MultiQueue/NetQueue
     ====================

This driver supports Rx/Tx netqueue pairs which can be allocated/freed dynamically by vmkernel.
The statistics for individual queues can be seen using below statistics command:
# vsish -e get /net/pNics/<vmnic_name>/stats

6.   Statistics
     ===========

Driver supports multiple statistics that can be fetched using below command:
#esxcli network nic stats get -n <vmnic_name>

Driver private statistics can be fetched using below command:
# vsish -e get /net/pNics/<vmnic_name>/stats

 
7.   Features under development
     ==========================

1. NCSI
2. OCBB/OCSD
3. Tx-switching: NPAR functions from same port won't ping each other.


8.   Contacting Support
     ===================

Please feel free to contact your QLogic approved reseller or QLogic 
Technical Support at any phase of integration for assistance. QLogic
Technical Support can be reached by the following methods: 

 Web:    http://support.qlogic.com
 E-mail: support@qlogic.com


(c) Copyright 2018. All rights reserved worldwide. QLogic, the QLogic 
logo, and the Powered by QLogic logo are registered trademarks of
QLogic Corporation. All other brand and product names are trademarks 
or registered trademarks of their respective owners.


