Menu

Lumen help

Connecting an Ubuntu bare metal server to Lumen Network Storage

Once you have configured your Lumen® Network Storage node, it must be connected to the bare metal server. Below are the steps and terminal commands necessary to mount the node to the server.  

Using a CentOS bare metal server? Learn how to connect a CentOS bare metal server to Network Storage.

To connect an Ubuntu bare metal server to Network Storage:

Step 1: Update the system and software

  1. Check for updates.
                apt-get update 

apt-get upgrade -y --auto-remove 

# apt-get dist-upgrade -y 

# apt-get autoremove -y 

# apt-get autoclean -y
  1. Install network tools.
                apt install inetutils-traceroute -y 

apt install traceroute -y
  1. Install NFS tools.
                apt install nfs-common -y
  1. Install iSCSI tools.
                apt install multipath-tools -y 

apt install open-iscsi -y 

apt install libiscsi-bin -y 

apt install tgt –y

Step 2: Set up /etc/hosts for Network Storage.

Example /etc/hosts for NFS ONLY:

 

                10.10.10.11      SNC01-NFS1 
10.10.10.12      SNC02-NFS2

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS

Example /etc/hosts for NFS and iSCSI:

                10.10.10.11      SNC01-NFS1 
10.10.10.12      SNC02-NFS2 
10.10.10.16      SNC01-ISCSI1 
10.10.10.15      SNC02-ISCSI1 
10.10.10.14      SNC01-ISCSI2 
10.10.10.13      SNC02-ISCSI2 

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS

Step 3: Update the /etc/netplan configuration.

Example: 

                cd /etc/netplan/ 
ls -al 
total 12 
drwxr-xr-x   2 root root 4096 Mar  1 17:56 . 
drwxr-xr-x 100 root root 4096 Mar  2 20:52 .. 
-rw-r--r--   1 root root  611 Mar  1 16:51 lumn.yaml 
     vlan.4: 
         id: 4 
         link: bond0 
         mtu: 8900 
         addresses: [10.10.10.21/24] 

Example of entire lumn.yaml: 
network: 
ethernets: 
     switchports: 
         match: 
             name: en* 
bonds: 
     bond0: 
         interfaces: [switchports] 
         mtu: 8900 
         parameters: 
             down-delay: 0 
             lacp-rate: fast 
             mii-monitor-interval: 0 
             mode: 802.3ad 
             transmit-hash-policy: layer2+3 
             up-delay: 0 
vlans: 
     vlan.3: 
         id: 3 
         link: bond0 
         mtu: 8900 
         addresses: [198.91.0.140/29] 
         gateway4: 198.91.0.137 
         nameservers: 
             addresses: 
             - 205.171.3.65 
             - 205.171.2.65 
     vlan.4: 
         id: 4 
         link: bond0 
         mtu: 8900 
         addresses: [10.10.10.21/24] 
#### 
#Try the netplan to verify functionality of network

netplan try
 
#Apply the netplan to make this permanent. 
netplan apply

Step 4: Verify Network Connectivity. Ping the local IP address, the NFS addresses, and the iSCSI addresses.

Example: 

                ping -c 3 LOCALHOST-LNS 

PING LOCALHOST-LNS (10.10.10.21) 56(84) bytes of data. 

64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=1 ttl=64 time=0.018 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=2 ttl=64 time=0.022 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=3 ttl=64 time=0.023 ms 

--- LOCALHOST-LNS ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2053ms 
rtt min/avg/max/mdev = 0.018/0.021/0.023/0.002 ms 

ping -c 3 SNC01-NFS1 

PING SNC01-NFS1 (10.10.10.11) 56(84) bytes of data. 

64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=1 ttl=64 time=10.7 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=2 ttl=64 time=0.153 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=3 ttl=64 time=0.093 ms 

--- SNC01-NFS1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2019ms 
rtt min/avg/max/mdev = 0.093/3.638/10.668/4.971 ms 

ping -c 3 SNC02-NFS2 

PING SNC02-NFS2 (10.10.10.12) 56(84) bytes of data. 

64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=1 ttl=64 time=10.6 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=2 ttl=64 time=0.124 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=3 ttl=64 time=0.099 ms 

--- SNC02-NFS2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2017ms 

rtt min/avg/max/mdev = 0.099/3.605/10.592/4.940 ms 

### Ping the iSCSI IPs 

ping -c 3 SNC01-ISCSI1 

PING SNC01-ISCSI1 (10.10.10.16) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=1 ttl=64 time=16.2 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=2 ttl=64 time=0.122 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=3 ttl=64 time=0.098 ms 

--- SNC01-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2011ms 
rtt min/avg/max/mdev = 0.098/5.474/16.203/7.586 ms 

ping -c 3 SNC02-ISCSI1 

PING SNC02-ISCSI1 (10.10.10.15) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=1 ttl=64 time=44.6 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=2 ttl=64 time=0.143 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=3 ttl=64 time=0.091 ms 

--- SNC02-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2031ms 
rtt min/avg/max/mdev = 0.091/14.956/44.635/20.985 ms 

ping -c 3 SNC01-ISCSI2 

PING SNC01-ISCSI2 (10.10.10.14) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=1 ttl=64 time=10.8 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=2 ttl=64 time=0.136 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=3 ttl=64 time=0.088 ms 

--- SNC01-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2010ms 
rtt min/avg/max/mdev = 0.088/3.663/10.766/5.022 ms 

ping -c 3 SNC02-ISCSI2 

PING SNC02-ISCSI2 (10.10.10.13) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=1 ttl=64 time=10.7 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=2 ttl=64 time=0.092 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=3 ttl=64 time=0.092 ms 
  
--- SNC02-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2005ms 
rtt min/avg/max/mdev = 0.092/3.615/10.662/4.982 ms

Step 5: Set up NFS.

  1. Display the available mounts. 
                showmount --exports <ip_address/Host_name of the nfs-server>

Example: 

                showmount --exports <ip_address/Host_name of the nfs-server> 

Example:  

showmount --exports SNC01-NFS1 
Export list for SNC01-NFS1: 
/LNSML4ECTST00009VOL01 (everyone) 
/LNSML4ECTST00009VOL02 (everyone) 
/                      (everyone) 

showmount --exports SNC02-NFS2 

Export list for SNC02-NFS2: 

/LNSML4ECTST00009VOL01 (everyone) 
/LNSML4ECTST00009VOL02 (everyone) 
/                      (everyone)
  1. Create mount points. This system has two mount points.
                mkdir -p /nfs1 

mkdir -p /nfs2
  1. Mount a NFS volume. 
                mount -t nfs -o options <nfs-server:/remote/share_directory>  	</local/mount_directory>

Example:

                mount -t nfs -o clientaddr=10.10.10.21 SNC01-NFS1:/LNSML4ECTST00009VOL01 /nfs1 

mount -t nfs -o clientaddr=10.10.10.21 SNC02-NFS2:/LNSML4ECTST00009VOL02 /nfs2
  1. Check for available space. 
                df –BG

Example:

                f -BG 

Filesystem                                     1G-blocks  Used Available Use% Mounted on 

<<SNIP>> 

SNC01-NFS1:/LNSML4ECTST00009VOL01       48G    1G       48G   1% /nfs1 

SNC02-NFS2:/LNSML4ECTST00009VOL02     4750G    1G     4750G   1% /nfs2
  1. Ensure the NFS mounts are writable. 
                cd /nfs1 

dd if=/dev/urandom of=./zz1 bs=10k count=10000 

ls -al 

cd /nfs2 

dd if=/dev/urandom of=./zz2 bs=100k count=10000 

ls –al
  1. Unmount the NFS mounts.
                cd /root 

umount /nfs1 

umount /nfs2
  1. Add NFS mount(s) to /etc/fstab to test functionality.

Example:

                SNC01-NFS1:/LNSML4ECTST00009VOL01 /nfs1 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0 

SNC02-NFS2:/LNSML4ECTST00009VOL02 /nfs2 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0
  1. Check FSTAB NFS entries using the mountpoint.

Example:

                mount /nfs1 

mount /nfs2 

df -BG 

Filesystem                                     1G-blocks  Used Available Use% Mounted on 

SNC01-NFS1:/LNSML4ECTST00009VOL01       48G    1G       48G   1% /nfs1 

SNC02-NFS2:/LNSML4ECTST00009VOL02     4750G    2G     4749G   1% /nfs2

Step 6: Set up iSCSI.

  1. Display your wwids.
                more /etc/multipath/wwids

Important: wwids will be empty immediately after setup.

  1. Check or fix multiplat.cond.
                more /etc/multipath/wwids

Example:

                defaults { 
    user_friendly_names yes 
}
  1. Set up iSCSI service.
                # Setup 
mkdir /iscsi0 
cd /iscsi0 
ls -al 
df -k 
cd 
systemctl restart iscsid.service
  1. To add a Network Storage iSCSI volume, you will need the InitiatorName for your system. This information is located in /etc/iscsi/initiatorname.iscsi.

Example:

                Use iqn.1993-08.org.debian:01:1928d48ec7e2
  1. Use the ign value in Network Storage for the iSCSI Create Column in the Mapped Client IQN field.

Important: Create a volume before proceeding.

  1. Discover the target.
                iscsiadm -m discovery -t sendtargets -p <IP Address/Hostname>

Example:

                SNC01-		ISCSI1 

10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI1 

10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC01-ISCSI2 

10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI2 

10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31
  1. List any blocked devices.
                lsblk –e7

Example:

                sblk -e7 

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT 

nvme0n1     259:0    0 447.1G  0 disk 
├─nvme0n1p1 259:1    0   512M  0 part /boot/efi 
└─nvme0n1p2 259:2    0  55.9G  0 part 
  └─os-root 253:0    0  55.9G  0 lvm  / 
nvme1n1     259:3    0 894.3G  0 disk 
nvme2n1     259:4    0 894.3G  0 disk
  1. Use iscsiadmin to list all target information.

Important: You may experience timeouts.

                iscsiadm --mode node -l all
  1. Restart iscsi.
                /etc/init.d/open-iscsi restart

Example:

                /etc/init.d/open-iscsi restart 

Restarting open-iscsi (via systemctl): open-iscsi.service.
  1. Run discovery using iscsiadm. 

Example:

                iscsiadm -m discovery -t sendtargets -p  

ML4ECTSTS0201SNC01-	ISCSI1 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI1 

10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC01-ISCSI2 

10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI2 

10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31
  1. Log in to each target name using the above returned targets from iscsiadm. 

Example:

                iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.13:3260,1036 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.15:3260,1039 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.16:3260,1037 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.14:3260,1035 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] successful.
  1. During the preceding process, the following message may appear:
                iscsiadm --mode node --targetname iqn.1992-	08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.13:3260,1036 --login 

iscsiadm: default: 1 session requested, but 1 already present. 

iscsiadm: Could not log into all portals

If the message appears, log out of each portal then log in to each again.

Example:

                iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.13:3260,1036 --logout 
Logging out of session [sid: 9, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] 
Logout of [sid: 9, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.15:3260,1039 --logout 
Logging out of session [sid: 12, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] 
Logout of [sid: 12, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.16:3260,1037 --logout 
Logging out of session [sid: 11, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] 
Logout of [sid: 11, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.14:3260,1035 --logout 
Logging out of session [sid: 10, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] 
Logout of [sid: 10, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] successful.
  1. Show all sessions.
                iscsiadm --mode session

Example:

                iscsiadm --mode session 

tcp: [17] 10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash) 
tcp: [18] 10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash) 
tcp: [19] 10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash) 
tcp: [20] 10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash)
  1. Display information about the nodes.
                iscsiadm -m node -o show | more
  1. Fix /etc/multipath.conf.
                vi /etc/multipath.conf

Example:

                defaults { 
    user_friendly_names yes 
} 
blacklist { 
    devnode "^(ram|zram|raw|loop|fd|md|dm-|sr|scd|dcssblk)[0-9]" 
    devnode "^(td|hd|vd)[a-z]" 
    devnode "^cciss!c[0-9]d[0-9]*" 
}
  1. Rescan iscsiadm.
                iscsiadm -m session --rescan

Example:

                iscsiadm -m session --rescan 

Rescanning session [sid: 17, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] 
Rescanning session [sid: 18, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] 
Rescanning session [sid: 19, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] 
Rescanning session [sid: 20, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260]
  1. Run iscsiadm discovery for all ST devices for each portal. 
                iscsiadm -m discovery -t st -p <portal> -o new

Example:

                iscsiadm -m discovery -t st -p 10.10.10.13,3260 -o new 

10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
iscsiadm -m discovery -t st -p 10.10.10.15,3260 -o new 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t st -p 10.10.10.16,3260 -o new 

10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t st -p 10.10.10.14,3260 -o new 

10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31
  1. Run iscsi-ls on each portal.
                iscsi-ls -s iscsi://<portal>

Example:

                iscsi-ls -s iscsi://10.10.10.13,3260 

Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
root@ML4ECTSTSB030206:~# iscsi-ls -s iscsi://10.10.10.15,3260 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
root@ML4ECTSTSB030206:~# iscsi-ls -s iscsi://10.10.10.16,3260 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
root@ML4ECTSTSB030206:~# iscsi-ls -s iscsi://10.10.10.14,3260 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035
  1. Verify the partitions.
                cat /proc/partitions
  1. Check the mulipaths.
                lsblk -e7 

NAME                                MAJ:MIN RM   SIZE RO TYPE  	MOUNTPOINT 

sda                                   8:0    0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
sdb                                   8:16   0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
sdc                                   8:32   0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
sdd                                   8:48   0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
nvme0n1                             259:0    0 447.1G  0 disk 
├─nvme0n1p1                         259:1    0   512M  0 part  	/boot/efi 
└─nvme0n1p2                         259:2    0  55.9G  0 part 
  └─os-root                         253:0    0  55.9G  0 lvm   / 
nvme1n1                             259:3    0 894.3G  0 disk 
nvme2n1                             259:4    0 894.3G  0 disk
  1. Check wwids.
                cat /proc/partitions

Example:

                more /etc/multipath/wwids 

# Multipath wwids, Version : 1.0 
# NOTE: This file is automatically maintained by multipath and multipathd. 
# You should not need to edit this file in normal circumstances. 
# 
# Valid WWIDs: 
/3600a098038314630452b532f46714a4c/   ### OLD WWID FOR DM-0 - IGNORE ### 
/3600a098038314630452b532f46714a4f/
  1. Mulipath the WWID.
                multipath -f /dev/mapper/<each lun>

Example:

                multipath -f /dev/mapper/3600a098038314630452b532f46714a4f
  1. Rest and list.
                multipath -r 

multipath -ll 

3600a098038314630452b532f46714a4f dm-1 NETAPP,LUN C-Mode 
size=250G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw 
|-+- policy='service-time 0' prio=50 status=active 
| |- 10:0:0:0 sda     8:0   active ready running 
| `- 13:0:0:0 sdd     8:48  active ready running 
`-+- policy='service-time 0' prio=10 status=enabled 
  |- 11:0:0:0 sdb     8:16  active ready running 
  `- 12:0:0:0 sdc     8:32  active ready running
  1. List the devices.
                ls -al /dev/dm-* 

brw-rw---- 1 root disk 253, 0 Mar  4 22:43 /dev/dm-0 

brw-rw---- 1 root disk 253, 1 Mar  7 21:05 /dev/dm-1
  1. Put the EXT4 filesystem on dm-1.
                mkfs -t ext4 /dev/dm-#

Example:

                mkfs -t ext4 /dev/dm-1 

mke2fs 1.45.5 (07-Jan-2020) 

Creating filesystem with 65536000 4k blocks and 16384000 inodes 
Filesystem UUID: 2b53636b-d450-4ba3-84ce-c74f4f9ce770 
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872 
Allocating group tables: done 
Writing inode tables: done 
Creating journal (262144 blocks): done 
Writing superblocks and filesystem accounting information: done
  1. Make a mountpoint.
                mkdir /iscsi1
  1. Add to fstab.
                more /etc/fstab 

/dev/dm-1 /iscsi1 ext4 defaults 0 0 

Example including both NFS mountpoints: 

SNC01-NFS1:/LNSML4ECTST00009VOL01 /nfs1 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0 
SNC02-NFS2:/LNSML4ECTST00009VOL02 /nfs2 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0 
# iSCSI for on-tap 
/dev/dm-1 /iscsi1 ext4 defaults 0 0
  1. Mount the volume. 
                mount /iscsi1
  1. Show the iscsi mounted volume
                df -BG 

Filesystem                                     1G-blocks  Used Available Use% Mounted on 

<<SNIP>> 

SNC01-NFS1:/LNSML4ECTST00009VOL01       48G    1G       48G   1% /nfs1 
SNC02-NFS2:/LNSML4ECTST00009VOL02     4750G    2G     4749G   1% /nfs2 
/dev/mapper/3600a098038314630452b532f46714a4f       246G    1G      233G   1% /iscsi1
  1. Write test data to /iscsi1.
                cd /iscsi1/ 

dd if=/dev/urandom of=./zz1 bs=1000k count=4000 

4000+0 records in 

4000+0 records out 
4096000000 bytes (4.1 GB, 3.8 GiB) copied, 28.1742 s, 145 MB/s 
# Note - /dev/urandom max is about 145 MB/s 
# After some other writes 
df -BG /iscsi1 
Filesystem                                    1G-blocks  Used Available Use% Mounted on 
/dev/mapper/3600a098038314630452b532f46714a4f      246G   43G      191G  19% /iscsi1