Fred’s Blog

Firefox does a lot of IO to the disk even thou you have lots of spare Ram, due to Sqlite, Bookmarks, History and Cache.

To make Firefox faster is to store the whole profile folder into a Ram Disk.
RamDisk in Linux are called TmpFS. You can also use shared memory folder /dev/shm if you have it in your fstab.

This post in the gentoo forums explains how to do it in Gentoo linux.

I made a similar script to make it work in OSX Leopard.

The Script have 2 parts, Start.sh and Stop.sh
Here are the Scripts:

Start.sh

#!/bin/bash
# Run this script to enable the Ramdisk for Firefox profiles
VolumeName="Mozilla"
 
# Size in MB, make sure is not too low or not too high
SizeInMB=220
 
NumSectors=$((2*1024*SizeInMB))
 
DeviceName=`hdid -nomount ram://$NumSectors`
 
echo $DeviceName
 
diskutil eraseVolume HFS+ RAMDisk $DeviceName
 
# move the current profiles folder
mv Profiles Profiles_ &&
 
# make a symlink to the ramdisk
ln -s /Volumes/RAMDisk ./Profiles &&
 
# then copy it to the ramdisk
/opt/local/bin/gcp -a Profiles_/* Profiles

Stop.sh

#!/bin/bash
cd ~/Library/Cache/Firefox/
 
# clean the cache
rm -rf  Profiles/*/Cache/* &&
 
# the rsync I use was installed from macports
# will save your modifications to the DISK
/opt/local/bin/rsync -av --delete ./Profiles/ ./Profile_/ &&
 
# sometimes during unmount it will say disk is in use.
umount /Volumes/RAMDisk &&
rm -rf Profiles &&
mv Profiles_ Profiles

You can also use ‘tar’ instead of ‘rsync’. I just love rsync more.

* Warning: The ramdisk contents will be erased after you umount the ramdisk.

Have fun.

Gentoo Linux ROCKS!

The OSX file vault feature will only let you activate it on your current home folder if you have a HFS+ case-insensitive file system. It will not let you activate it on a current HFS case-sensitive.

But there is a trick, when creating a “new” clean user it will allow you to enable filevault for that new user.

So, here is how you do it.

But before you try, make sure you have enough free space, try to get rid of huge files and folders, backup your data. (in this case, we will copy your data, so you will have 2 copies for safety)

also clean your applications cache, such as firefox, camino, etc…:

$ cd ~/Library/Caches

$ find ./ -name “Cache” -exec rm -rf {} \;

Steps

1. Create another admin user, for example “admin”, with administrator privileges

2. login as that user.

3. move your old user folder.

$ cd /Users/

$ sudo mv myusername myusername.bak

3. delete your old user from the “Accounts” preferences pane.

4. then create it again and check the option to use filevault

5. Logout from “admin” and login again as newuser.

6. Now copy the old data to your home folder. ( will take a very long time for that)

$ sudo cp -a /Users/myusername.bak/* ~/

$ sudo chown -R myusername ~/

7. Logout and login again,

8. If everything looks fine, delete the backup folder

$ sudo rm -rf /Users/myusername.bak

if you have any confusions let me know in comments.

First of all, what do I mean by safe sleep, fast sleep or hibernate?

Safe sleep is the way OSX sleeps to RAM and as well create a sleepimage (which is the size of your RAM). In case you run out of battery, so you can still resume from the image if the battery is dead, (and you have plugged it in :)  Provides very fast wake up, uses the battery while sleeping.

Fast Sleep is just sleep to RAM, same as safe sleep but no image creation, and if your battery is dead, the mac will cold boot. Provides very fast wake up same as safe sleep, uses the battery while sleeping.

Hibernate is when it uses the sleepimage all the time. Slower sleep and slow wake up, but it does not use battery at all…

So…

I found in this website http://alt.cc/jk/2007/08/07/safe-sleep-addendum/ this nice script that handles when to FastSleep or when to Hibernate.

you can get safe sleep with the command “$ sudo pmset hibernate 3″

I have modified the script it little :

#!/bin/sh
MODE=`/usr/bin/pmset -g | grep hibernatemode | awk '{ print $2 }'`
LEFT=`/usr/bin/pmset -g batt | grep Internal | awk '{ print $2 }' | awk -F % '{ print $1 }'`
HIBERNATE=20
FASTSLEEP=50
echo "Running safesleep.sh => MODE: ${MODE} LEFT: ${LEFT}" >> /var/log/system.log
if [ $LEFT -lt $HIBERNATE ] && [ $MODE != 3 ] ; then
	{
		echo "Less than ${HIBERNATE}% remains" >> /var/log/system.log
		echo "Setting Hibernate (hibernate mode 1)" >> /var/log/system.log
		`/usr/bin/pmset -a hibernatemode 1`
		LS=`ls -al /private/var/vm/sleepimage`
		echo "The sleepimage should be created:" >> /var/log/system.log
		echo "${LS}" >> /var/log/system.log
	}
elif  [ $LEFT -gt $FASTSLEEP ] && [ $MODE != 0 ]; then
	{
		echo "Greater than ${FASTSLEEP}% remains" >> /var/log/system.log
		echo "Setting FastSleep (hibernate mode 0)" >> /var/log/system.log
		`/usr/bin/pmset -a hibernatemode 0`
		`rm -rf /var/vm/sleepimage`
	}
fi

save it as /Users/your_user_name/.crons/safesleep.sh

now make it permanent in a crontab to run every 10 minutes, but wait, anacron (osx default cron) only supports daily/weekly/monthly jobs! (unless you patch it)

I guess we’ll have to install fcron:

$ sudo port -d install fcron

$ sudo vi /opt/local/etc/fcrontab


@,runas(root) 10 sh /Users/your_user_name/.crons/safesleep.sh


$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.fcron.plist

$ sudo launchctl start org.macports.fcron

that’s it.

enjoy

I just found the new Virtual Machine Software released by SUN Microsystems. It’s called VirtualBox.

http://virtualbox.org

VirtualBox runs on Windows, Linux, Macintosh and OpenSolaris hosts and supports a large number of Guest Operating systems.

Im running windows XP on my macbook pro and it seems ok, thou OSX is really slow, when I run the virtual machine. (and yes, I install virtual box drivers and tools on the guest)

here is some VirtualBox Goodnes that explains baout the command line tool:

http://dkprojects.wordpress.com/2008/10/29/more-virtualbox-goodness-tackling-the-cli/

Suppose you have a Model called Article that contains a text field and a format field.

You would like to use haml, textile or HTML to edit your Article from the admin interface.

  create_table "articles", :force => true do |t|
    t.string   "title"
    t.text     "body"
    t.string   "formatting_type", :limit => 20, :default => "HTML"
  end

app/models/article.rb

It’s quite simple. All you have to do is to add this helper in your application_helper.rb

  def print_formated(type,text)
    case type
    when "HTML"
      text
    when "Plain Text"
      h text
    when "HAML"
      Haml::Engine.new(text).render
    when "Syntaxy"
      Syntaxi.line_number_method = 'none'
      Syntaxi.new(text).process
    when "Textile"
      RedCloth.new(text).to_html
    end
  end

In your views/articles/_form.haml add the select field.

  = label :article, :formatting_type
  = select(:article, :formatting_type, Article::FORMATTING_TYPES.collect {|p| p }, { :include_blank => false })

Then in the Show view (articles/show.haml)

 
.article
  .title
    %h1
      = h @article.title
  .body
    = print_formated(@article.formatting_type, @article.body)

that’s pretty much it.

To install the great Mod_Rails on Gentoo linux it’s as easy as 5 steps.

Since you are Gentoo user, i don’t need to go to details. You know what you doing.

1. Recompile Apache non-threaded

add this to /etc/portage/package.use

www-servers/apache -threads

and this to /etc/make.conf

APACHE2_MPMS="prefork"

2. Re emerge apache

# emerge -va apache

3. Passenger is in gentoo portage, but its in testing. Currently Version 2.0.1

# echo "www-apache/passenger" >> /etc/portage/package.keywords

4. Install Passenger

# emerge -va passenger

If it tries to install rails 2.0.2, rake, and lots of other gems that you already have installed trough rubygems, then run emerge with –nodeps option

# emerge -va --nodeps passenger

5. Edit /etc/conf.d/apache and add “-D PASSENGER” to apache options

for example mine looks like this:

APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D LANGUAGE -D PROXY -D PASSENGER"

That’s it.

Now just drop a similar vhost config file inside /etc/apache/vhosts.d/

This is a sample vhost file for a rails app.

<VirtualHost *:80>
  ServerName mydomain.com
  DocumentRoot /myapp/public
  Include /etc/apache2/vhosts.d/deflate.conf
  RailsBaseURI /
  # The maximum number of Ruby on Rails application instances that may be simultaneously active.
  # A larger number results in higher memory usage, but improved ability to handle concurrent HTTP clients.
  # normally 1 to 10. (1 for each 50mb ram)
  RailsMaxPoolSize 1
  # The maximum number of seconds that a Ruby on Rails application instance may be idle.
  # That is, if an application instance hasn't done anything after the given number of seconds,
  # then it will be shutdown in order to conserve memory. ( 1 hour)
  RailsPoolIdleTime 3600
  RailsEnv 'production'
  <Directory /myapp/public>
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
    </Directory>
</VirtualHost>

My sample deflate.conf,
used to gzip the content

<Location />
	SetOutputFilter DEFLATE
	#
	# Netscape 4.x has some problems...
	BrowserMatch ^Mozilla/4 gzip-only-text/html
	#
	# Netscape 4.06-4.08 have some more problems
	BrowserMatch ^Mozilla/4\.0[678] no-gzip
	#
	# MSIE masquerades as Netscape, but it is fine
	BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
	# NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
	# the above regex won't work. You can use the following
	# workaround to get the desired effect:
	BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
	# Don't compress images
	SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
	SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
	SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
	# Make sure proxies don't deliver the wrong content
	Header append Vary User-Agent env=!dont-vary
</Location>

DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio
LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate
CustomLog /var/log/apache2/deflate_log deflate

* Update on July 10, 2008.

- Now using gentoo portage to install it. it’s more smooth.

Note:

Personally I found that Thin + nginx uses less memory (Nginx 3MB, thin ~56MB)
apache + passenger will use quite more. (Apache 40MB, mod_rails ~50MB)

The other day I had to set up a VPS machine at Slicehost for a client on a tight budget. I paid for 256mb VPS based on Gentoo, my distro of choice.

But 256MB of ram? what can you do with just 256?

Normally a default 256mb linux machine would not handle very well a set of Apache + Mysql + 1 mongrel/thin/ebb instance. due to the high memory usage of a default configuration, it will swap very often.

After much research and instinct i made it run one thin servers with mysql and nginx, without any swapping, and really fast as it can be.

If your linux start swapping often your performance will go down to the floor… Swapping is bad, specially on a XEN VPS.

The trick is to setup Mysql to use MYISAM and use Nginx instead of apache.

Here is the process list with the Resident Memory usage, after 30 days uptime and about 1,800 page views on the website.

and others, such as sshd, cron, iptables, bash, together about 5mb.

As you can see, total of memory usage of the applications on the server is about 83 MB, thus leaving the server with 170MB of ram for the linux itself and file cache.

this is what #free command tells:

             total       used       free     shared    buffers     cached
Mem:           256        235         20          0         54         62
-/+ buffers/cache:        128        128
Swap:          511          0        511

Nice uh?
you can also make use of the nice tool called “vmstat”
it’s very import that ’si’ (swap in) and ’so’ (swap out) stays zero all the time.

i.e. running vmstat 10 times with a 4 seconds interval. (ignore the 1st line)

# vmstat 4 10
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0    116  14900  56668  62692    1    1     3     3    9    5  0  0 100  0
 0  0    116  14908  56668  62692    0    0     0     0   34   53  0  0 100  0
 0  0    116  14968  56668  62692    0    0     0     0   37   54  0  0 100  0
 0  0    116  14968  56668  62692    0    0     0     0   31   52  0  0 100  0
....

Here is the recipe:

1. Use MyIsam instead of InnoDB

You can read about it more in here:

http://blog.evanweaver.com/articles/2007/04/30/top-secret-tuned-mysql-configurations-for-rails/

I forgot to add that you need to dump your database first:

mysqldump -u root –all-databases > dump.sql

then change my.cnf accordingly,

restart mysql and reload the database

mysql -u root < dump.sql

Change only the values for my.cnf as shown below, and delete all innodb related stuff

# can be safely set to 1M if you are really tight on Ram
key_buffer 			       = 4M

max_allowed_packet 			= 1M
table_cache 				= 32
sort_buffer_size 			= 512k
net_buffer_length 			= 8K
read_buffer_size 			= 512k
read_rnd_buffer_size 		= 512K

# can be safely set to 1M
myisam_sort_buffer_size 	= 2M  

language 					= /usr/share/mysql/english

# security:
# using "localhost" in connects uses sockets by default
# skip-networking
bind-address				= 127.0.0.1

# No logging,
# make sure you backup your database more often.
#log-bin

server-id 					= 1

# point the following paths to different dedicated disks
tmpdir 						= /tmp/

# Very important to have this here.
# otherwise it will still load InnoDB.
skip-innodb

[mysqldump]
quick
max_allowed_packet 			= 16M

[mysql]
# uncomment the next directive if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer 					= 8M
sort_buffer_size 			= 8M
read_buffer 				= 2M
write_buffer 				= 2M

[myisamchk]
key_buffer 					= 8M
sort_buffer_size 			= 8M
read_buffer 				= 2M
write_buffer 				= 2M

[mysqlhotcopy]
interactive-timeout

If you get problems reloading the database, stop mysql delete the contents in /var/lib/mysql/* , then run mysql-installdb and start it and reload again the sql dump file.

Actually that’s the way i most prefer..

2. Use nginx

this is an example nginx config file, located at /etc/nginx/nginx.conf

user nginx nginx;
# set to 2 or 3 if you have more processors or cores.
# it will use about 3MB per worker
worker_processes 1;

error_log /var/log/nginx/error_log info;

events {
	# default is 8192
	worker_connections  1024;
	use epoll;
}

http {
	include		/etc/nginx/mime.types;
	default_type	application/octet-stream;

	log_format main
		'$remote_addr - $remote_user [$time_local] '
       	'"$request" $status $bytes_sent '
		'"$http_referer" "$http_user_agent" '
		'"$gzip_ratio"';

	client_header_timeout	10m;
	client_body_timeout	10m;
	send_timeout		10m;
        client_body_buffer_size    128k;
        proxy_buffer_size          4k;
        proxy_buffers              4 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;

	connection_pool_size		256;
	client_header_buffer_size	1k;
	large_client_header_buffers	4 2k;
	request_pool_size		4k;

	gzip on;
	gzip_http_version 1.1;
        gzip_comp_level 6;
        gzip_proxied any;
        gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
        gzip_buffers 16 8k;
        # Disable gzip for certain browsers.
        gzip_disable "MSIE [1-6]\.(?!.*SV1)";
	gzip_min_length	1100;

	output_buffers	1 32k;
	postpone_output	1460;

	sendfile	on;
	tcp_nopush	on;
	tcp_nodelay	on;

	keepalive_timeout	75 20;

	ignore_invalid_headers	on;

	index index.html;

    # The following includes are specified for virtual hosts
    include /var/www/apps/bla.com/shared/config/nginx.conf;

}

this is an example vhost file

upstream mongrel_bla_com {
  server 127.0.0.1:8001;
}

server {
	listen 80;
	client_max_body_size 40M;
	server_name bla.com www.bla.com;
	root /var/www/apps/bla.com/current/public;
	access_log  /var/www/apps/bla.com/shared/log/nginx.access.log;

	location / {
                proxy_set_header  X-Real-IP  $remote_addr;
                proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect false;
                if (-f $request_filename/index.html) {
                        rewrite (.*) $1/index.html break;
                }
                if (-f $request_filename.html) {
                        rewrite (.*) $1.html break;
                }
                if (!-f $request_filename) {
                        proxy_pass http://mongrel_bla_com;
                        break;
                }
	}

	location ~* ^.+\.(jpg|js|jpeg|png|ico|gif|txt|js|css|swf|zip|rar|avi|exe|mpg|mp3|wav|mpeg|asf|wmv)$ {
		root /var/www/apps/bla.com/current/public;
	}

}

3. Remove unnecessary Services you dont need.

Some linux distros have enabled by default services we dont need.

such as cupsd, apmd, acpid, mdns, samba, nfs, ftpd… etc…

This is my make.conf in case it helps

Note that I set MAKEOPTS=”-J1″ , it will only use 1 gcc process at the time, and not disturb the system, (machine has 4 cores)

Also portage_niceness to 18, to make sure it will run smooth and not disturb thin and mysql.

from nice man page: “Nicenesses range from -20 (most favorable scheduling) to 19 (least favorable).”

CFLAGS="-march=athlon64 -O2"
CHOST="x86_64-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j1"

USE="3dnow 3dnowext apache2 \
     bash-completion bzip2 \
     -cups \
     gzip httpd hpn \
     innodb imagemagick \
     javascript jpeg \
     mmx mmxext mysql \
     nptl nptlonly \
     perl phyton png \
     ruby \
     screen sse sse2 sqlite sqlite3 ssl \
     threads udev unicode utf8 \
     vim-syntax \
     -X -kde -gnome -gtk -bindist \
     xml xml2 zlib "

GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo http://gentoo.cites.uiuc.edu/pub/gentoo/ ftp://gentoo.mirrors.tds.net/gentoo http://ge
APACHE2_MPMS='worker'
PORTAGE_NICENESS=18

if you want to use mod_rails Passenger, set APACHE2_MPMS=’prefork’

note: I am positive you can throw in another thin server instance, and it will still not swap, or swap very little at all.

have fun

**************************

Updates :

Wanna know what Slicehost Manager Diagnostics says about my VPS ?

Diagnostics

    * Your slice is currently running.
    * The host server is up.
    * Your swap IO usage over the last 4 hours is low: 0.0016 reads/s, 0.0 writes/s. (Read more about swap here)
    * Your root IO usage over the last 4 hours is low: 0.038 reads/s, 0.1643 writes/s.
    * The host server's load is nominal: 0.00, 0.03, 0.00.
  

Tommy has just released an new mysql-ruby package.

Actually 2 of them:

mysql-ruby-2.7.5 and mysql-ruby-2.8pre2

They are Ruby 1.9 compatible
Requirements

* MySQL 5.0.51a
* Ruby 1.8.6, 1.9.0

here is the link http://tmtm.org/en/mysql/ruby/

Great Job

Encrypt folders in Mac OSX with encfs

OSX already include the File Vault functionality that allows you to encrypt your whole Home Folder.
Thou the storage overhead is so small, the time to encrypt it the first time is very very long.
if you have Videos, and big files, it takes even longer.

What if I don’t want to encrypt my big folders like Movies, Music, Pictures, Pdfs?

I only want to encrypt my Documents folder.
Be aware that VMWARE stored the virtual machine files under this folder, you should move it to outside Documents.

WARNING:

Be careful with this tutorial,
Write down your password somewhere and BACKUP your data before going further these steps.

if you forget your password, say good bye to your data.

THERE IS NO WAY TO GET YOUR DATA BACK!!!

TOOLS required:


# update your ports to get the latest encfs that runs ok on OSX10.5
$ sudo port selfupdate

# install encfs
$ sudo port install encfs

or Download macfuse and encfs from google:

http://code.google.com/p/macfuse/

and

http://code.google.com/p/encfs/

Lets move The Documents folder contents to another folder:


$ cd
$ mkdir temp_documents
$ mv Documents/* temp_documents/


Create the directory to hold the encrypted files, it can be any name.

Run this only one time. The first time to setup the folder…

$ mkdir .documents


Setup the encryption

$ encfs ~/.documents/ ~/Documents/


you will see this:

fred@Macintosh ~ $ encfs ~/.documents/ ~/Documents/
Creating new encrypted volume.
Please choose from one of the following options:
enter "x" for expert configuration mode,
enter "p" for pre-configured paranoia mode,
anything else, or an empty line will select standard mode.

now, after you pass this step, the file system will be mounted as well.

encfs uses FuseFS, so it behaves just like a mount point

to unmount it you do


$ unmount ~/Documents


to mount it again issue this command:

$ encfs ~/.documents/ ~/Documents/
# or this way, which will look with better names and a folder icon on Desktop:
$ encfs ~/.documents/ ~/Documents/ -- -o fsname=Documents -o volname=Documents -o local


to check mounted filesystems


$ mount


you should be able to see:


encfs@fuse2 on /Users/fred/Documents (fusefs, nodev, nosuid, synchronous, mounted by fred)


or this if you used the longer command.

Documents on /Users/fred/Documents (fusefs, local, nodev, nosuid, synchronous, mounted by fred)


Now, with the encrypted folder “mounted”, mv the data from that temp folder to the new encrypted folder:

WARNING: be carefull here


$ cp temp_documents/* Documents/
$ rm -rf temp_documents/


that’s it folks.

Final overview:


# to create the encrypted folder:
$ encfs ~/.documents/ ~/Documents/
#to Mount it (enable)
$ encfs ~/.documents/ ~/Documents/
#or
$ encfs ~/.documents/ ~/Documents/ -- -o fsname=Documents -o volname=Documents -o local
#to Umount it (disable)
$ umount ~/Documents


don’t change anything inside .documents
remember the dot in the front means the folder is invisible
you won’t see it in Finder.

This also should work for Linux.