When running a lot of unix commands via scripts, it’s quite easy to hit the maximum allowable processes per user, or the maximum number of open files per user.
My OS 10.6.8 mac has the following limits defined.
lovebox-4:[~] $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 256 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 266 virtual memory (kbytes, -v) unlimited
A user can increase these limits directly from the shell using the ulimit command.
ulimit -n
Changes the maximum allowable number of concurrently open files.
ulimit -u
Changes the maximum allowable concurrent processes.
These limits are in place to stop a user from hurting the overall system for other users. Typically on an OS X system – there is only one user, so it’s quite OK to increase the limits. Another thing to know is that the user can only increase the limit up to the maximum allowed by the system. This system maximum can also be changed (by the system administrator / root) – again on most macs that will be the same user. So how to do this?.
First, let’s see what the system limits are using the command
sysctl
lovebox-4:[~] $ sysctl -a | egrep '(maxfiles|maxproc)' kern.maxproc = 532 kern.maxfiles = 12288 kern.maxfilesperproc = 102480 kern.maxprocperuid = 266 kern.maxproc: 532 kern.maxfiles: 12288 kern.maxfilesperproc: 20480 kern.maxprocperuid: 512
Now, I want to change those system maximums, so that I can change the user maximums too.
sudo sysctl -w kern.maxfilesperproc=20480 kern.maxfilesperproc: 10240 -> 20480
sudo sysctl -w kern.maxprocperuid=512 kern.maxprocperuid: 266 -> 512
So, now I have higher system limits, I need to tell OS X that I want to use those larger limits. Again, the system allows the user to protect himself by restricting himself to a lower limit.
Next, open .bashrc and put the new limits in there
ulimit -n 1024 ulimit -u 512
You may not be able to set the new limits directly in an existing shell even though the kernel maximums were changed. However a newly executed shell should have the new limits
e.g. In an existing shell
lovebox-4:[~] $ ulimit -n 1024 -bash: ulimit: open files: cannot modify limit: Operation not permitted
In a newly created shell window
lovebox-4:[~] $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 512 virtual memory (kbytes, -v) unlimited
So, here’s something non-intuative
lovebox-4:[~] $ ulimit -n 1024 <--- Works because we set 1024 in bashrc lovebox-4:[~] $ ulimit -n 512 <---- Lower the limit lovebox-4:[~] $ ulimit -n 1024 <--- Now it cannot be raised in this shell, or any descendents. -bash: ulimit: open files: cannot modify limit: Operation not permitted
Now, that's sorted out our shells. To do a similar thing but for processes invoked from 'finder' e.g. Web browser or something. We need to mess with launchd...
lovebox-4:[~] $ sudo launchctl limit cpu unlimited unlimited filesize unlimited unlimited data unlimited unlimited stack 8388608 67104768 core 0 unlimited rss unlimited unlimited memlock unlimited unlimited maxproc 266 532 maxfiles 256 unlimited
lovebox-4:[~] $ sudo launchctl limit maxproc 1024
lovebox-4:[~] $ sudo launchctl limit
cpu unlimited unlimited
filesize unlimited unlimited
data unlimited unlimited
stack 8388608 67104768
core 0 unlimited
rss unlimited unlimited
memlock unlimited unlimited
maxproc 1024 1024
maxfiles 256 unlimited
^This may not be persistent, and will only be picked up when new processes are launched. In other words, if an already executing process has reached the maximum number of open files – doing the above will not help until the process is re-started.