David Pattersons seminal paper on RAID

• A Case for Redundent Arrays of Inexpensive Disks (RAID)

A Fast File System for UNIX

I am mentoring a new starter at NetApp and so I found a couple of papers which discuss at a high level some of the problems that WAFL and RAID set out to solve. Both papers are quite old, but are interesting in that they discuss the big picture of the original problem.

A Storage Networking Appliance
This is a great paper which discusses the principles of the “filer” concept.

• NetApp Internal (Updated 2006) :

File System Design for an NFS
File Server Appliance

• File System Design for an NFS
  File Server Appliance

• Usenix : Text file (1994 Version):
• ACM Digital Library PDF (1994 Version) :

Just remember, that when writing self contained awk scripts, the command line interpreter line (i.e. the #! line) needs to have -f at the end. Otherwise you will get all sorts of divide by zero errors, and other similarly non-helpful messages.

#!/bin/awk -f

#!/opt/local/bin/gawk -f

Just recently an ex-colleague from Sun asked me to send him some background info on NetApp filer performance. Normally the filer can just be plugged in and it ought to be ready to go without much tweaking. The following documents give some background on how to setup a filer in various environments.

• A list of all technical reports
• Database Performance with NFS: Optimizing Oracle on NFS.
  A little dated, but still an interesting introduction to using NFS in an enterprise environment

• Best Practices for File System Alignment in Virtual Environments. Some specific tips for using a filer with hypervisors
• Evaluating the Performance of FCoE, iSCSI, and FC Using DSS Workloads with Microsoft SQL Server 2008
• If You’re Doing This, Then Your Storage Could Be Underutilized
• Introduction to Predictive Cache Statistics
  DataONTAP contains a facility which enables users to determine how much benefit they might get from installing a flash based accelerator card (FlashCache).

• Performance Study of IBM DB2 9 with NFS, iSCSI, and FCP
  The recommendations in this document are also generally applicable across NFS, iCSCI and FCP.

• NetApp and VMware Virtual Infrastructure 3 Storage Best Practices

Courtesy of Cem Kaner you are employed as a performance analyst, then the following probably applies to you too. I consider performance defects as a class of bug, which need to be fixed by some other entity (maybe a programmer, or an architect or sysadmin). This means you’re about to create work for some other person, who might prefer to be watching Star Wars, drinking coffee or spending time with their family.

1. The point of testing is to find bugs.

2. Bug reports are your primary
work product. This is what people outside
of the testing group will most notice and most
remember of your work.

3. The best tester isn’t the one who finds the most bugs or who embarrasses the most programmers. The best tester is the one who gets the most bugs fixed.

4. Programmers operate under time constraints and competing priorities. For example, outside of the 8- hour workday, some programmers prefer sleeping and watching Star Wars to fixing bugs.

A bug report is a tool that you use to sell the
programmer on the idea of spending her time
and energy to fix a bug.

Go here for the full text.. I
———————————————————————–

More of an overview than an anything specific. Makes a good point on the provenance of the eventual vizualization ( I have started to include pointers to the R / gnuplot script as a comment in wiki pages, when I include a chart) .

Hadley Wickham at Google (Data analysis with R and ggplot2)

For some reason the device /dev/rawctl has a habit of disappearing. This can be fixed by creating the device file /dev/rawctl with the major/minor number 162,0

mknod /dev/rawctl c 162 0
 

This will allow /dev/raw devices to be created. However /dev/raw/raw0 cannot be used since 162,0 is used by /dev/rawctl.

So, assume I have a git repository on an iDisk, mounted to my mac a /Volumes/garyjlittle. I also have a dropbox account mounted to /Dropbox.

I have a directory called ~/scripts on several machines (home and work) that I keep synchronized and version contolled using git.

Now, I want to move a git repository called “scripts” from iDisk to Dropbox.

Here’s what I do. I have a subdir called ‘git’ on my Dropbox drive, and within there I have a bunch of ‘bare’ repositories – because I always make edits in ~/scripts. The directory /Dropbox/git/scripts.git is just there to be a shared resource to push/pull from.

Make the directory hierarchy in Dropbox.

lovebox:[~] $ mkdir /Dropbox/git
lovebox:[~] $ mkdir /Dropbox/git/scripts.git

Change directory to the Dropbox script directory we just created.

lovebox:[~] $ cd /Dropbox/git/scripts.git
 

Now inititalize that directory as a git bare repository.

lovebox:[/Dropbox/git/scripts.git] $ git init --shared --bare

Now go to the iDisk directory which currently holds the bare repository

lovebox:[~] $ cd /Volumes/garyjlittle/scripts

And push over the repository to the new Dropbox location.

lovebox:[/Volumes/garyjlittle/scripts] $ git push --mirror /Dropbox/git/scripts.git/

Counting objects: 49, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (49/49), done.
Writing objects: 100% (49/49), 21.52 KiB, done.
Total 49 (delta 18), reused 0 (delta 0)
Unpacking objects: 100% (49/49), done.
To /Dropbox/git/scripts.git/
 * [new branch]      master -> master

Now we have to tell each remote location that the shared repository has changed. It’s easiest to just rename the old directory from ~/scripts to ~/old.scripts. And then do a git clone from the new Dropbox git location.

lovebox:[~] $ mv scripts/ old.scripts

lovebox:[~] $ git clone /Dropbox/git/scripts.git/ scripts
Initialized empty Git repository in /Users/gjl/scripts/.git/
lovebox:[~] $ cd scripts

You might notice (as I did) that there are fewer scripts in the new ~/scripts directory. That’s probably because not all of the scripts really were under git control. This could be a good time for housekeeping. Add in the scripts that you really want to keep under git version control, and get rid of the crufty old one-time stuff that inevitably builds up over time.

Page http://omniti.com/surge/2010/speakers/neil-gunther

Slides http://s.omniti.net/surge/i/content/slides/NeilGunther.pdf

Neil Gunther on Scalability at SurgeCon 2010

Terrible audio, but worth listening to.