I guess it works!

http://now.netapp.com/NOW/cgi-bin/simulator

I am using a Linux virtual machine inside Virtual Box, so I connect to that Linux machine and copy over the tarfile, that I downloaded to my mac. An alternative would be do download it from within the VM.

I am going to download the tarfile to /var/tmp, and then do the actual install to /opt/sim.

[root@localhost tmp]# pwd
/var/tmp

Here I am copying the tarfile to the /var/tmp dir.

[root@localhost tmp]# scp gjl@192.168.56.1:~/software/7.3.2-tarfile-v22.tar .
7.3.2-tarfile-v22.tar                                               100%   56MB  14.1MB/s   00:04
[root@localhost tmp]# ls -l
total 57572
-rw-r--r--. 1 root root 58951680 2010-01-30 12:45 7.3.2-tarfile-v22.tar

Now I can un-tar the file (it was already uncompressed).

[root@localhost tmp]# tar xf 7.3.2-tarfile-v22.tar

The tarfile extracts a directory named “simulator” inside this dir is the setup tool

[root@localhost tmp]# cd simulator/

[root@localhost simulator]# pwd
/var/tmp/simulator
[root@localhost simulator]# ls
disks2.tgz  doc          readme.htm  setup.sh  Vmware, Linux and Simulator installation.doc
disks.tgz   license.htm  runsim.sh   sim.tgz

Now i run the setup.sh script

[root@localhost simulator]# ./setup.sh 

Script version 22 (18/Sep/2007)
Where to install to? [/sim]: /opt/sim
Would you like to install as a cluster? [no]:
Would you like full HTML/PDF FilerView documentation to be installed [yes]: no
Continue with installation? [no]: yes
Creating /opt/sim
Unpacking sim.tgz to /opt/sim
Configured the simulators mac address to be [00:50:56:14:bd:36]
Please ensure the simulator is not running.
Your simulator has 2 disk(s). How many more would you like to add? [0]: 12

The following disk types are available in MB:
        Real (Usable)
  a -   43   ( 14)
  b -   62   ( 30)
  c -   78   ( 45)
  d -  129   ( 90)
  e -  535   (450)
  f - 1024   (900)

If you are unsure choose the default option a
What disk size would you like to use? [a]:
Disk adapter to put disks on? [0]:
Use DHCP on first boot? [yes]: no
Ask for floppy boot? [no]:
Your default simulator network interface is already configured to eth1.
Which network interface should the simulator use? [eth1]:
Your system has 435MB of free memory. The smallest simulator memory you should choose is 110MB. The maximum simulator memory is 395MB.
The recommended memory is 512MB.
Your original default appears to be too high. Seriously consider adjusting to below the maximum amount of 395MB.
How much memory would you like the simulator to use? [512]: 350
Create a new log for each session? [no]:
Overwrite the single log each time? [yes]:
Adding 12 additional disk(s).
Complete. Run /opt/sim/runsim.sh to start the simulator.

Since I answered no to the DHCP question, I will need to configure an IP address when I startup the simulator. Let’s do that.

[root@localhost simulator]# /opt/sim/runsim.sh

Here is a copy of my session.

runsim.sh script version Script version 22 (18/Sep/2007)
This session is logged in /opt/sim/sessionlogs/log

NetApp Release 7.3.2: Thu Oct 15 03:13:41 PDT 2009
Copyright (c) 1992-2009 NetApp.
Starting boot on Sat Jan 30 17:53:32 GMT 2010
Sat Jan 30 17:54:17 GMT [iomem.card.fail.fpga.rev:CRITICAL]: Acceleration card (model name X1938A-R5, serial number 5012345670, part number virgo-simulator) on slot 1 has firmware version 1.0 (200905011442), requires 1.1 (200908030903).
Sat Jan 30 17:54:17 GMT [iomem.init.fail:CRITICAL]: Failed to initialize acceleration card (model name X1938A-R5, serial number 5012345670, part number virgo-simulator) in slot 1.
Sat Jan 30 17:54:24 GMT [fmmb.current.lock.disk:info]: Disk v4.16 is a local HA mailbox disk.
Sat Jan 30 17:54:24 GMT [fmmb.current.lock.disk:info]: Disk v4.17 is a local HA mailbox disk.
Sat Jan 30 17:54:24 GMT [fmmb.instStat.change:info]: normal mailbox instance on local side.
Sat Jan 30 17:54:26 GMT [raid.cksum.replay.summary:info]: Replayed 0 checksum blocks.
Sat Jan 30 17:54:26 GMT [raid.stripe.replay.summary:info]: Replayed 0 stripes.
sparse volume upgrade done. num vol 0.
Vdisk Snap Table for host:0 is initialized
Sat Jan 30 17:54:29 GMT [vol.language.unspecified:info]: Language not set on volume vol0. Using language config "C". Use vol lang to set language.
Sat Jan 30 17:54:29 GMT [rc:notice]: The system was down for 8809411 seconds
Sat Jan 30 17:54:29 GMT [useradmin.added.deleted:info]: The role 'compliance' has been added.
Sat Jan 30 17:54:30 GMT [useradmin.added.deleted:info]: The group 'Backup Operators' has been modified.
/etc/rc is missing. Running configuration dialog.

	NetApp Release 7.3.2: Thu Oct 15 03:13:41 PDT 2009
	System ID: 0099910788 ()
	System Serial Number: 987654-32-0 ()
	System Storage Configuration: Multi-Path
	System ACP Connectivity: No Connectivity
	Model Name: Simulator
	Processors: 1
	slot 0: NetApp Virtual SCSI Host Adapter v0
                14 Disks:              0.6GB
		2 shelves with LRC
	slot 1: NetApp Virtual SCSI Host Adapter v1
	slot 2: NetApp Virtual SCSI Host Adapter v2
	slot 3: NetApp Virtual SCSI Host Adapter v3
	slot 4: NetApp Virtual SCSI Host Adapter v4
                14 Disks:              0.6GB
		2 shelves with LRC
	slot 5: NetApp Virtual SCSI Host Adapter v5
	slot 6: NetApp Virtual SCSI Host Adapter v6
	slot 7: NetApp Virtual SCSI Host Adapter v7
	slot 8: NetApp Virtual SCSI Host Adapter v8
                4 Tapes:            VT-100MB
                                    VT-100MB
                                    VT-100MB
                                    VT-100MB
Please enter the new hostname []:

From here on, the process is the same as installing a new filer.

Please enter the new hostname []: pocketsim
Do you want to enable IPv6? [n]:
Do you want to configure virtual network interfaces? [n]:
Please enter the IP address for Network Interface ns0 []: 192.168.56.12
Please enter the netmask for Network Interface ns0 [255.255.255.0]:
Please enter media type for ns0 {100tx-fd, auto} [auto]:
Please enter the IP address for Network Interface ns1 []:
Would you like to continue setup through the web interface? [n]:
Please enter the name or IP address of the IPv4 default gateway: 192.168.56.1
	The administration host is given root access to the filer's
	/etc files for system administration.  To allow /etc root access
	to all NFS clients enter RETURN below.
Please enter the name or IP address of the administration host: 192.168.56.1
Please enter timezone [GMT]: EST
Where is the filer located? []: LinuxVM
What language will be used for multi-protocol files (Type ? for list)?:
language not set
Do you want to run DNS resolver? [n]:
Do you want to run NIS client? [n]:
	The Shelf Alternate Control Path Management process provides the ability
	to recover from certain SAS shelf module failures and provides a level of
	availability that is higher than systems not using the Alternate Control
	Path Management process.
Do you want to configure the Shelf Alternate Control Path Management interface for SAS shelves [n]:
Setting the administrative (root) password for pocketsim ...
New password:
Retype new password:

Do you want to configure the Shelf Alternate Control Path Management interface for SAS shelves [n]:
Setting the administrative (root) password for pocketsim …
New password:
Retype new password:
Sat Jan 30 12:56:41 EST [passwd.changed:info]: passwd for user ‘root’ changed.
Sat Jan 30 12:56:42 EST [dfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk drives
Sat Jan 30 12:56:42 EST [sfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk shelves.
Sat Jan 30 12:56:42 EST [tapemc.alias.addOK:info]: Alias st0 automatically added for tape device WWN[0:042:424200:000000].
Sat Jan 30 12:56:42 EST [tapemc.alias.addOK:info]: Alias st1 automatically added for tape device WWN[0:142:424200:000000].
Sat Jan 30 12:56:42 EST [tapemc.alias.addOK:info]: Alias st2 automatically added for tape device WWN[0:242:424200:000000].
Sat Jan 30 12:56:42 EST [tapemc.alias.addOK:info]: Alias st3 automatically added for tape device WWN[0:342:424200:000000].
Sat Jan 30 12:56:42 EST [netif.linkUp:info]: Ethernet ns0: Link up.
Sat Jan 30 12:56:42 EST [perf.archive.start:info]: Performance archiver started. Sampling 22 objects and 195 counters.
add net default: gateway 192.168.56.1
There are 12 spare disks; you may want to use the vol or aggr command
to create new volumes or aggregates or add disks to the existing aggregate.
Sat Jan 30 12:56:43 EST [snmp.agent.msg.access.denied:warning]: Permission denied for SNMPv3 requests from root. Reason: Password is too short (SNMPv3 requires at least 8 characters).
Sat Jan 30 12:56:44 EST [rc:info]: Registry is being upgraded to improve storing of local changes.
Sat Jan 30 12:56:44 EST [rc:info]: Registry upgrade successful.
Sat Jan 30 12:56:45 EST [mgr.boot.disk_done:info]: NetApp Release 7.3.2 boot complete. Last disk update written at Tue Oct 20 13:50:57 EST 2009
Sat Jan 30 12:56:45 EST [mgr.boot.reason_ok:notice]: System rebooted after a halt command.
This process will enable CIFS access to the filer from a Windows(R) system.
Use “?” for help at any prompt and Ctrl-C to exit without committing changes.

Your filer does not have WINS configured and is visible only to
clients on the same subnet.
Do you want to make the system visible via WINS? [n]:
A filer can be configured for multiprotocol access, or as an NTFS-only
filer. Since multiple protocols are currently licensed on this filer,
we recommend that you configure this filer as a multiprotocol filer

(1) Multiprotocol filer
(2) NTFS-only filer

Selection (1-2)? [1]: 1
CIFS requires local /etc/passwd and /etc/group files and default files
will be created. The default passwd file contains entries for ‘root’,
‘pcuser’, and ‘nobody’.
Enter the password for the root user []:
Retype the password:
The default name for this CIFS server is ‘POCKETSIM’.
Would you like to change this name? [n]:
Data ONTAP CIFS services support four styles of user authentication.
Choose the one from the list below that best suits your situation.

(1) Active Directory domain authentication (Active Directory domains only)
(2) Windows NT 4 domain authentication (Windows NT or Active Directory domains)
(3) Windows Workgroup authentication using the filer’s local user accounts
(4) /etc/passwd and/or NIS/LDAP authentication

Selection (1-4)? [1]: 4
What is the name of the Workgroup? [WORKGROUP]:
CIFS – Starting SMB protocol…
Welcome to the WORKGROUP Windows(R) workgroup

CIFS local server is running.

Password:
pocketsim> Sat Jan 30 12:57:41 EST [console_login_mgr:info]: root logged in from console

pocketsim>

Now I have installed the filer, I can connect to it from outside the Linux VM. In this example "lovebox" is the name of my MacOS laptop (which hosts the Linux VM).

lovebox:~ gjl$ telnet 192.168.56.12
Trying 192.168.56.12...
Connected to 192.168.56.12.
Escape character is '^]'.

Password:
pocketsim> Sat Jan 30 12:58:47 EST [telnet_0:info]: root logged in from host: 192.168.56.1

pocketsim>
pocketsim>

Most of the tools used to collect network trace data support the ’snaplen’ option which controls how much data to capture per packet. For NFS traces a snaplen of 256 bytes seems to work well. With a snaplen of 256 the trace analysers will have enough data to reconstruct the filehandles, offset and IO size. Tools like tshark will be able to match the NFS response to the request and so you can see the response time from the NFS server.

In this trace below, the total response time is from the first NFS READ Call (packet 13) until the final RPC Continuation in the sequence. In NFS the NFS V3 READ Reply (Packet 14) is just the beginning of the NFS server response. The bulk of the payload is in the “RPC Continuation” packets. The number of Continuation packets will depend on the size of the NFS request and the size of the Ethernet frames. In the trace below, the IO size is 32K, the Ethernet segment size is 1500 and the IO stream is single threaded.

I use the tshark option “-R rpc” to show the “NFS READ calls” as well as the “RPC Continuation” responses from the NFS server.

The command line.

lovebox:test_snaplen gjl$ tshark -r snap256_from_open -R rpc|head -35

The trace.

 13   3.529598 192.168.10.3 -> 192.168.10.4 NFS V3 READ Call, FH:0x44281a4d Offset:0 Len:32768
 14   3.529844 192.168.10.4 -> 192.168.10.3 NFS V3 READ Reply (Call In 13) Len:32768[Unreassembled Packet]
 15   3.529856 192.168.10.4 -> 192.168.10.3 RPC Continuation
 17   3.529872 192.168.10.4 -> 192.168.10.3 RPC Continuation
 18   3.529882 192.168.10.4 -> 192.168.10.3 RPC Continuation
 20   3.529896 192.168.10.4 -> 192.168.10.3 RPC Continuation
 21   3.529905 192.168.10.4 -> 192.168.10.3 RPC Continuation
 23   3.529921 192.168.10.4 -> 192.168.10.3 RPC Continuation
 24   3.529930 192.168.10.4 -> 192.168.10.3 RPC Continuation
 26   3.529944 192.168.10.4 -> 192.168.10.3 RPC Continuation
 27   3.529955 192.168.10.4 -> 192.168.10.3 RPC Continuation
 29   3.529970 192.168.10.4 -> 192.168.10.3 RPC Continuation
 30   3.529980 192.168.10.4 -> 192.168.10.3 RPC Continuation
 32   3.529997 192.168.10.4 -> 192.168.10.3 RPC Continuation
 33   3.530005 192.168.10.4 -> 192.168.10.3 RPC Continuation
 35   3.530019 192.168.10.4 -> 192.168.10.3 RPC Continuation
 36   3.530029 192.168.10.4 -> 192.168.10.3 RPC Continuation
 38   3.530044 192.168.10.4 -> 192.168.10.3 RPC Continuation
 39   3.530053 192.168.10.4 -> 192.168.10.3 RPC Continuation
 41   3.530068 192.168.10.4 -> 192.168.10.3 RPC Continuation
 42   3.530077 192.168.10.4 -> 192.168.10.3 RPC Continuation
 44   3.530092 192.168.10.4 -> 192.168.10.3 RPC Continuation
 45   3.530100 192.168.10.4 -> 192.168.10.3 RPC Continuation
 47   3.530115 192.168.10.4 -> 192.168.10.3 RPC Continuation
 48   3.530197 192.168.10.3 -> 192.168.10.4 NFS V3 READ Call, FH:0x44281a4d Offset:32768 Len:32768

If you are processing someone elses trace data, one clue that the snaplen was too short is that the NFS reads do not contain the offset or Len values. In this example the snaplen was 192. Notice that packet 27 does not show the offset of the read, compare to packet 13 in the above trace, where the snaplen was 256.

 17  30.646984 192.168.10.3 -> 192.168.10.4 NFS V3 ACCESS Call, FH:0x34d96998
 18  30.647233 192.168.10.4 -> 192.168.10.3 NFS V3 ACCESS Reply (Call In 17)
 20  30.647279 192.168.10.3 -> 192.168.10.4 NFS V3 GETATTR Call, FH:0x44281a4d
 21  30.647472 192.168.10.4 -> 192.168.10.3 NFS V3 GETATTR Reply (Call In 20)  Regular File mode:0666 uid:0 gid:0
 22  30.647501 192.168.10.3 -> 192.168.10.4 NFS V3 ACCESS Call, FH:0x44281a4d
 23  30.647727 192.168.10.4 -> 192.168.10.3 NFS V3 ACCESS Reply (Call In 22)
 27  30.648254 192.168.10.3 -> 192.168.10.4 NFS V3 READ Call[Packet size limited during capture]
 28  30.648539 192.168.10.4 -> 192.168.10.3 NFS V3 READ Reply (Call In 27) Len:32768[Packet size limited during capture]

In wireshark, there are a number of settings which control how wireshark/tshark attempts to re-assemble rpc/tcp/nfs packets. Enabling “more aggressive” RPC decoding seemed to mess up the ability to show the RPC continuation packets. Here are the RPC settings from the wireshark preferences used to generate this data. These prefs also affect the way tshark works.

lovebox:test_snaplen gjl$ grep rpc ~/.wireshark/preferences
dcerpc.desegment_dcerpc: TRUE
dcerpc.reassemble_dcerpc: TRUE
rpc.desegment_rpc_over_tcp: TRUE
rpc.defragment_rpc_over_tcp: TRUE
rpc.max_tcp_pdu_size: 4194304
rpc.dissect_unknown_programs: FALSE
rpc.find_fragment_start: FALSE
rpcap.desegment_pdus: TRUE
rpcap.decode_content: TRUE
rpcap.linktype: 0

Slides

In the past, the name of the disk would be displayed in format, something like “SUN 10G blah blah” but sometimes the disk name is just “DEFAULT” which can be annoying. It turns out that it is quite simple to change the name of the disk, IF you happen to know the correct sequence.

So, in this case, I want to change the name of disk 1, which currently has the name “DEFAULT” followed by some details of the physical layout.

Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c3d0 DEFAULT cyl 4092 alt 2 hd 128 sec 32
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0
       1. c3d1 DEFAULT cyl 141 alt 2 hd 64 sec 32
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@1,0

The first thing I am going to do is to create a file which contains the geometry of this disk. In the case of x86 accessing the disk via a BIOS, the “disk” is actually just the partition which is given over to Solaris, but in principle we think of it as “the disk”

We will use the format “save” option from within the format/disk mentu to generate the file. The file is called format.dat by default, and this will be a freshly created file, similar in syntax to /etc/format.dat, but only containing information about one disk (the one which we want to change).

Specify disk (enter its number): 1
selecting c3d1
Controller working list found
[disk formatted, defect list found]

FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        fdisk      - run the fdisk program
        repair     - repair a defective sector
        show       - translate a disk address
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        save       - save new disk/partition definitions
        volname    - set 8-character volume name
        !     - execute , then return
        quit
format> save
Saving new disk and partition definitions
Enter file name["./format.dat"]:
format> q

Next we will edit that file, and change the name to whatever it is we want to use. The original content of format.dat

gjl@vbox-sol1:/a# cat format.dat
#
# New disk/partition type  saved on Thu Jan  7 11:14:21 2010
#
disk_type = "DEFAULT" \
         : ctlr = ata : ncyl = 141 : acyl = 2 : pcyl = 143 \
         : nhead = 64 : nsect = 32 : rpm = 3600

partition = "original" \
         : disk = "DEFAULT" : ctlr = ata \
         : 2 =  0, 288768 : 8 =  0, 2048 : 9 =  1, 4096
 

and now the edited version, I have hust change the word “DEFAULT” to “SOLARIS_DATA_VDI”

gjl@vbox-sol1:/a# cat format.dat
#
# New disk/partition type  saved on Thu Jan  7 11:14:21 2010
#
disk_type = "SOLARIS_DATA_VDI" \
         : ctlr = ata : ncyl = 141 : acyl = 2 : pcyl = 143 \
         : nhead = 64 : nsect = 32 : rpm = 3600

partition = "original" \
         : disk = "SOLARIS_DATA_VDI": ctlr = ata \
         : 2 =  0, 288768 : 8 =  0, 2048 : 9 =  1, 4096

So, now I want to use this file to change the values stored on the disk, which is used by format to identify the disk. To do that I will be using the -x option to format, and point to my just edited copy of format.dat

gjl@vbox-sol1:/a# format -x ./format.dat
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c3d0 DEFAULT cyl 4092 alt 2 hd 128 sec 32
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0
       1. c3d1 DEFAULT cyl 141 alt 2 hd 64 sec 32
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@1,0

Notice that nothing has changed in the output of format…. yet. Now I will tell format that I want to change the disk type, and the disk type “SOLARIS_DATA_VDI” will now appear in the menu. It has all the same values as originally, except that I have changed the name.

First I select the disk I am going to change (disk 1)

Specify disk (enter its number): 1
selecting c3d1
Controller working list found
[disk formatted, defect list found]

Then I select “type” to change the type and my new disk type appears as a selectable option.

format> type

AVAILABLE DRIVE TYPES:
        0. SOLARIS_DATA_VDI
        1. DEFAULT
        2. DEFAULT

I select Option ‘0′

Specify disk type (enter its number)[2]: 0
selecting c3d1
No current partition list
No defect list found
[disk formatted, no defect list found]

and finally I label the disk.

format> la
Ready to label disk, continue? y

format> q
gjl@vbox-sol1:/a#

Now, when I run format again, I see that the disk type/name has changed, even though I am no longer using my special “format.dat”. Which is because the name is stored on the disk itself.

gjl@vbox-sol1:/a# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c3d0 DEFAULT cyl 4092 alt 2 hd 128 sec 32
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0
       1. c3d1 SOLARIS_DATA_VDI cyl 141 alt 2 hd 64 sec 32
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@1,0

Of course, a simpler way to mark the disk would be just to add a “Volume name” to the disk. I specifically wanted to get rid of the disk type “DEFAULT” from my disk list, so changing the volume name alone was not enough. In the format output below, I create the volume name “VDISK” which appears in addition to the name “SOLARIS_DATA_VDI” which is actually the ASCII disk name/type”

format> vol
Enter 8-character volume name (remember quotes)[""]:"VDISK"
Ready to label disk, continue? y

format> di

AVAILABLE DISK SELECTIONS:
       0. c3d0 DEFAULT cyl 4092 alt 2 hd 128 sec 32
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0
       1. c3d1 SOLARIS_DATA_VDI cyl 141 alt 2 hd 64 sec 32  VDISK
          /pci@0,0/pci-ide@1,1/ide@0/cmdk@1,0

Here’s the link: Velocity 2009

Some snaps from the top of Mt. Occoneechee. Click “Read more” to see slideshow.

PDF

The original link to arXiv.org (whatever that is)