Issue sync first, to flush dirty pages back to the backing store

sync

Issue this command to invalidate the cache

echo 3 > /proc/sys/vm/drop_caches

This kernel.org page has an interesting list of the various settable values for the linux page cache. http://www.kernel.org/doc/Documentation/sysctl/vm.txt

drop_caches

Writing to this will cause the kernel to drop clean caches, dentries and
inodes from memory, causing that memory to become free.

To free pagecache:
	echo 1 > /proc/sys/vm/drop_caches
To free dentries and inodes:
	echo 2 > /proc/sys/vm/drop_caches
To free pagecache, dentries and inodes:
	echo 3 > /proc/sys/vm/drop_caches

I need to send a MIME encoded email ( a png file) to myself. uuencoded files no longer play well with Outlook 2011 on Mac (no idea why). Thankfully we can replace mailx with mutt to do the MIME encoding for us

diskperf-3650-2:[/tmp] $ mutt -s “From mutt” -a /u/little/charts/spc1_csc_cam.png little@netapp.com

Mutt requires the < /dev/null to avoid going into interactive mode. Presumably it checks to see if input is from stdin and goes interactive if it is. The redirection could also point to some boilerplate text “This is the chart you requested” but redirecting

After some connectivity swap-a-roos in the lab, I could no longer see my LUNS from the linux host attached to my filer.

In this case I am using a QLogic HBA – and I am not using any of the NetApp host side tools – just the sanlun tool.

Using the SANsurfer Menu (/opt/QLogic_Corporation/SANsurferCLI) I can tell that this linux host can see the filers’ LUNS over FC. But there are no SCSI /dev/sdX devices for them, and so Linux cannot use them…

Here’s how I checked to see that there was FC connectivity – which also confirms that the FC protocol is working.

	SANsurfer FC/CNA HBA CLI

	v1.7.2 Build 7

    Main Menu

    1:	General Information  <---- Option 1
    2:	HBA Information
    3:	HBA Parameters
    4:	Target/LUN List
    5:	iiDMA Settings
...

    General Information Menu

    1:	Host Information
    2:	Host Topology
    3:	Report     <---- Option 3..
    4:	Refresh
    5:	Return to Previous Menu

	Note: 0 to return to Main Menu
	Enter Selection: 1

   Report Menu

    HBA Model QLE2462
      1: Port   1: WWPN: 21-00-00-E0-8B-9B-C5-36 Online
      2: Port   2: WWPN: 21-01-00-E0-8B-BB-C5-36 Online
      3: All HBAs  <---- Option 3
      4: Return to Previous Menu

	Note: 0 to return to Main Menu
	Enter Selection: 3

I could see that there was connectivity from the Linux host to the filer

---------------------------------------
LUN 1
---------------------------------------
Product Vendor                    : NETAPP
Product ID                        : LUN
Product Revision                  : 811a
LUN                               : 1
Size                              : 17.93 GB
Type                              : SBC-2 Direct access block device
			           (e.g., magnetic disk)
WWULN                             : 4E-45-54-41-50-50-20-20-20-4C-55-4E-20-32-46-68
			           72-53-3F-2D-68-4F-79-6C-33-00-00-00-00-00-00-00
OS LUN Name                       :

From the filer side, I could see that the host's FC adapters had connected to the filer,
and were in the right igroup

filer1*> igroup show
    filer1 (FCP) (ostype: linux):
        21:00:00:e0:8b:9b:c5:36 (logged in on: 0a)
        21:01:00:e0:8b:bb:c5:36 (logged in on: 0b)

The only thing that was missing was that there were no 'sd' devices created in Linux for these devices.

"sanlun" utility was not helpful and just told me that there wer no LUNs mapped.

The solution was to issue this very odd looking command

linuxhost:[/sys/class/scsi_host] $ echo "- - -" > host0/scan

This caused the sd devices to be created, representing the NetApp LUNs which I knew could already be seen over FC. Since I have both ports on the same HBA attached to the filer, host0 scan created my /dev/sdc* devices, and host1/scan created my /dev/sdd* devices.

The shell 'hung' for the duration of the command, and I would expect that Linux was off in kernel land for some time - and so i would NOT recommend issuing the command on a production server.

I'm still puzzled why the linux host did not see the luns even after reboot though.

Just remember, that when writing self contained awk scripts, the command line interpreter line (i.e. the #! line) needs to have -f at the end. Otherwise you will get all sorts of divide by zero errors, and other similarly non-helpful messages.

#!/bin/awk -f

#!/opt/local/bin/gawk -f

For some reason the device /dev/rawctl has a habit of disappearing. This can be fixed by creating the device file /dev/rawctl with the major/minor number 162,0

mknod /dev/rawctl c 162 0
 

This will allow /dev/raw devices to be created. However /dev/raw/raw0 cannot be used since 162,0 is used by /dev/rawctl.

Sizes larger than rsize result in parallel IO’s

64K Rather than 32K rsize gives ??? improvement (depends on userland read size no?)

Install ttcp on the Mac

lovebox-3:~ gjl$ sudo port install ttcp
--->  Fetching ttcp
--->  Attempting to fetch ttcp.c from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/ttcp
--->  Attempting to fetch ttcp.1 from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/ttcp
--->  Attempting to fetch README from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/ttcp
--->  Verifying checksum(s) for ttcp
--->  Extracting ttcp
--->  Configuring ttcp
--->  Building ttcp
--->  Staging ttcp into destroot
--->  Installing ttcp @1.10_0
--->  Activating ttcp @1.10_0
--->  Cleaning ttcp

Install nttcp on Linux

gjl@gjl-desktop:~$ sudo apt-get install nttcp
[sudo] password for gjl:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  openbsd-inetd inet-superserver
The following NEW packages will be installed:
  nttcp
0 upgraded, 1 newly installed, 0 to remove and 51 not upgraded.
Need to get 25.4kB of archives.
After this operation, 98.3kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ lucid/multiverse nttcp 1.47-12 [25.4kB]
Fetched 25.4kB in 0s (56.6kB/s)
Selecting previously deselected package nttcp.
(Reading database ... 243192 files and directories currently installed.)
Unpacking nttcp (from .../nttcp_1.47-12_i386.deb) ...
Processing triggers for man-db ...
Setting up nttcp (1.47-12) ...

Usage

On the server

gjl@gjl-desktop:~$ nttcp -i -p 5001

On the client

lovebox-3:~ gjl$ ttcp -s -t 10.0.1.4 -v -l 4096 -p 5002

Output

ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 10.0.1.4
ttcp-t: socket
ttcp-t: connect
ttcp-t: 16777216 bytes in 6.41 real seconds = 2554.32 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 3.21, calls/sec = 319.29
ttcp-t: 0.0user 0.2sys 0:06real 3% 0i+0d 202752maxrss 0+2pf 6593+53csw

Wifi

lovebox-3:~ gjl$ ttcp -s -t 10.0.1.4 -v -l 4096 -p 5002
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 10.0.1.4
ttcp-t: socket
ttcp-t: connect
ttcp-t: 16777216 bytes in 12.77 real seconds = 1282.91 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 6.39, calls/sec = 160.36
ttcp-t: 0.0user 0.2sys 0:12real 1% 0i+0d 202752maxrss 0+3pf 5549+69csw

5Ghz network

lovebox-3:~ gjl$ ttcp -s -t 10.0.1.4 -v -l 4096 -p 5002
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 10.0.1.4
ttcp-t: socket
ttcp-t: connect
ttcp-t: 16777216 bytes in 5.31 real seconds = 3083.91 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 2.66, calls/sec = 385.49
ttcp-t: 0.0user 0.1sys 0:05real 3% 0i+0d 202752maxrss 0+2pf 1185+318csw
lovebox-3:~ gjl$ ttcp -s -t 10.0.1.4 -v -l 4096 -p 5002

After a Windows reboot cygwin became quite broken, gcc would not work
for instance. It turns out that somehow the mounts became broken.
Here are the mounts that should be seen by a user.

$ cat mount.out
C:/cygwin/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin/lib on /usr/lib type ntfs (binary,auto)
C:/cygwin on / type ntfs (binary,auto)

Note how /bin and /lib are re-mounted on /usr/bin and /usr/lib. No
idea why that is.

To setup a default gateway as 10.199.50.1 use the following (note the final ’1′ at the end of the command line.

filer-3170> route add default 10.199.50.1 1

Today I was trying to run a packet trace to capture iSCSI traffic from a Windows box to one of our filers. The trace would only show iSCSI Read’s and no iSCSI Writes, even though I knew that there was write traffic. A packet trace taken from the filer using pktt showed the Reads and Writes I expected – which indicated something strange at the Windows end. It turned out that the cause was TCP/IP offloading. The intel NIC was configured to do the offloading, and evidently ‘hid’ some of the network activity from windump (a tcpdump implementation on Windows). Once I turned off the offloading from inside Windows using the “Advanced” Tab inside the NIC Properties window I was able to see the iSCSI reads and Writes at the Windows end.

My best guess is that the issue is to do with where in the network stack the WinPCAP module is inserted. It seems that some traffic was routed ‘around’ wherever in the stack WinPCAP was listening.

I recently setup cygwin on a Windows 2008 server, and when I tried to issue an rsh command, the rshd process was returning a NULL to the rsh client. It turned out that I had several copies of inetd running. Some I had started using cygrunsrv – and others using iu-config.

After killing all the copies of inetd, and starting a new one using cygrunsrv – everything was OK.

The key is to not run cygrunsrv, before setting up the cywgin “user” using iu-config – otherwise you end up with an inetd process owned by SYSTEM, and that – it seems is bad. When all is working, my ps -ef looks like this, where the user cyg_srv is the user setup during iu-config.

$ ps -ef

     UID     PID    PPID TTY     STIME COMMAND

Administ    3948       1 con  17:52:52 /usr/bin/bash

cyg_serv    3492       1   ?  17:55:22 /usr/bin/cygrunsrv

cyg_serv    3652    3492   ?  17:55:22 /usr/sbin/inetd

Administ    1308    3948 con  17:59:49 /usr/bin/ps

Tweet