about memory usage / WAS: ellinika sto xmms

Thu Jul 13 19:38:55 EEST 2006

On 2006-07-13 14:32, Nick Demou <ndemou at gmail.com> wrote:
>Την 13/7/2006, Giorgos Keramidas <keramida at ceid.upatras.gr> έγραψε:
>>On 2006-07-13 12:21, Nick Demou <ndemou at gmail.com> wrote:
>>> Την 13/7/2006, Christos Ricudis <ricudis at komodino.itc.auth.gr> έγραψε:
>>> > [...]
>>> >
>>> > An metrhseis to Resident Set Size (sthlh RSS sto ps -auxw) eisai panw
>>> > katw mesa.
>>>
>>> 4 what is worth: διαβάζοντας το paper του de icaza & Berkman που
>>> ανέφερα, η άποψη τους είναι πως μια καλή κατα προσέγγιση εκτίμηση για
>>> το πόση μνημη απαιτεί ένα προγραμμα προκύπτει αν από το RSS αφαιρέσεις
>>> το SHARE
>>
>> Ανάλογα τι είναι το 'SHARE'.
> 
> IMHO κατα 99% αναφέρεται στο SHR (πρέπει να διαβάσεις το εν λόγο paper
> [1] για να είσαι σίγουρος)
> 
>     SHR  --  Shared Mem size (kb)
>     The amount of shared memory used by a task.  It simply reflects 
>     memory that could be potentially shared with other processes.
> 
> [1] http://sunsite.bilkent.edu.tr/pub/linux/gnome/GDP/papers/whitepapers/MemoryUsage/MemoryUsage.ps

Den exei 100% adiko, alla akoma kai afairwntas tis perioxes mnhmhs pou
ena programma exei `mapped' logw kapoias biblio8hkhs, den einai panta
eykolo na breis akribws poso *fysikh* mnhmh xrhsimopoiei.

As, poume apo to /proc tou FreeBSD mporeis na breis tis perioxes pou
exei mapped to shell sou (oi ps-style headers exoun proste8ei apo mena,
ki epekshghsh katw apo to output):

,-----------------------------------------------------------------------
| $ cat /proc/$$/map
| START      END        RES PRES OBJ        PERM REF SCNT FLAGS   COW NC  TYPE    PATH
| 0x08048000 0x080d2000 131    0 0xc3ab3e88 r-x  12     6 0x0000  COW NC  vnode   /usr/local/bin/bash
| 0x080d2000 0x080d7000   5    0 0xc6cb69d8 rw-   1     0 0x2180  COW NNC vnode   /usr/local/bin/bash
| 0x080d7000 0x080db000   4    0 0xc57733c0 rw-   1     0 0x2180  COW NNC default -
| 0x080db000 0x08400000 105    0 0xc680a690 rwx   1     0 0x2180  COW NNC default -
| 0x280d2000 0x280f6000  26    0 0xc103ab40 r-x 137    59 0x0004  COW NC  vnode   /libexec/ld-elf.so.1
| 0x280f6000 0x280f8000   2    0 0xc62719d8 rw-   1     0 0x2180  COW NC  vnode   /libexec/ld-elf.so.1
| 0x280f8000 0x280fd000   4    0 0xc37014b0 rw-   1     0 0x2180  COW NC  default -
| 0x280fd000 0x28105000   8    0 0xc5e67690 rwx   1     0 0x2180  COW NC  default -
| 0x28105000 0x2813a000  50    0 0xc103a780 r-x  56    36 0x0004  COW NC  vnode   /lib/libncurses.so.6
| 0x2813a000 0x2813b000   1    0 0xc60db618 r-x   1     0 0x2180  COW NC  vnode   /lib/libncurses.so.6
| 0x2813b000 0x28144000   9    0 0xc6923438 rwx   1     0 0x2180  COW NC  vnode   /lib/libncurses.so.6
| 0x28144000 0x2814b000   7    0 0xc3ab4348 r-x  57    38 0x0000  COW NC  vnode   /usr/local/lib/libintl.so.6
| 0x2814b000 0x2814c000   1    0 0xc523f7f8 r-x   1     0 0x2180  COW NC  vnode   /usr/local/lib/libintl.so.6
| 0x2814c000 0x2814d000   1    0 0xc6111348 rwx   1     0 0x2180  COW NC  vnode   /usr/local/lib/libintl.so.6
| 0x2814d000 0x28156000   3    0 0xc53e6bb8 rwx   1     0 0x2180  COW NC  default -
| 0x28156000 0x28241000  35    0 0xc3909a50 r-x  70    40 0x0000  COW NC  vnode   /usr/local/lib/libiconv.so.3
| 0x28241000 0x28242000   1    0 0xc53d6528 r-x   1     0 0x2180  COW NC  vnode   /usr/local/lib/libiconv.so.3
| 0x28242000 0x28243000   1    0 0xc528a780 rwx   1     0 0x2180  COW NC  vnode   /usr/local/lib/libiconv.so.3
| 0x28243000 0x28319000 139    0 0xc103a618 r-x 190   114 0x0004  COW NC  vnode   /lib/libc.so.7
| 0x28319000 0x2831a000   1    0 0xc5dd1b40 r-x   1     0 0x2180  COW NC  vnode   /lib/libc.so.7
| 0x2831a000 0x28320000   6    0 0xc54bfac8 rwx   1     0 0x2180  COW NC  vnode   /lib/libc.so.7
| 0x28320000 0x28336000  10    0 0xc5eae258 rwx   1     0 0x2180  COW NC  default -
| 0xbfbe0000 0xbfc00000   7    0 0xc6c991e0 rwx   1     0 0x2180  COW NNC default -
| $
`-----------------------------------------------------------------------

Oi sthles einai:

    START	arxikh dieu8unsh mnhmhs ths vm perioxhs
    END		telikh dieu8unsh mnhmhs ths vm perioxhs
    RES		resident page count
    PRES	private resident page count
    OBJ		vm_object_t virtual address (vm internal)
    PERM	permissions
    REF		vm object reference count
    SHADOW	vm object "shadow" count
    FLAGS	vm object flags
    COW		cow flag
    NC		needs copy flag
    TYPE	vm object type
    PATH	vnode path

To synolo twn bytes pou einai mapped gia ayto to bash(1) process einai:

,-----------------------------------------------------------------------
| $ cat /proc/$$/map | awk '{print $1,$2}'
| 0x8048000 0x80d2000
| 0x80d2000 0x80d7000
| 0x80d7000 0x80db000
| 0x80db000 0x8400000
| 0x280d2000 0x280f6000
| 0x280f6000 0x280f8000
| 0x280f8000 0x280fd000
| 0x280fd000 0x28105000
| 0x28105000 0x2813a000
| 0x2813a000 0x2813b000
| 0x2813b000 0x28144000
| 0x28144000 0x2814b000
| 0x2814b000 0x2814c000
| 0x2814c000 0x2814d000
| 0x2814d000 0x28156000
| 0x28156000 0x28241000
| 0x28241000 0x28242000
| 0x28242000 0x28243000
| 0x28243000 0x28319000
| 0x28319000 0x2831a000
| 0x2831a000 0x28320000
| 0x28320000 0x28336000
| 0xbfbe0000 0xbfc00000
| $ cat /proc/$$/map | awk '{print $1,$2}' | sed -e 's/0x//g' | \
|       tr '[[:lower:]]' '[[:upper:]]' | awk '{print  $2"-"$1}'
| 80D2000-8048000
| 80D7000-80D2000
| 80DB000-80D7000
| 8400000-80DB000
| 280F6000-280D2000
| 280F8000-280F6000
| 280FD000-280F8000
| 28105000-280FD000
| 2813A000-28105000
| 2813B000-2813A000
| 28144000-2813B000
| 2814B000-28144000
| 2814C000-2814B000
| 2814D000-2814C000
| 28156000-2814D000
| 28241000-28156000
| 28242000-28241000
| 28243000-28242000
| 28319000-28243000
| 2831A000-28319000
| 28320000-2831A000
| 28336000-28320000
| BFC00000-BFBE0000
| $ ( echo 'ibase=16' ; \
|     cat /proc/$$/map | awk '{print $1,$2}' | sed -e 's/0x//g' | \
|         tr '[[:lower:]]' '[[:upper:]]' | awk '{print $2"-"$1}' ) \
|   | bc
| 565248
| 20480
| 16384
| 3297280
| 147456
| 8192
| 20480
| 32768
| 217088
| 4096
| 36864
| 28672
| 4096
| 4096
| 36864
| 962560
| 4096
| 4096
| 876544
| 4096
| 24576
| 90112
| 131072
| $  echo $( ( echo 'ibase=16' ; \
|              cat /proc/$$/map | awk '{print $1,$2}' | sed -e 's/0x//g' | \
|                  tr '[[:lower:]]' '[[:upper:]]' | \
|                  awk '{print $2"-"$1}' ) \
|            | bc ) | sed -e 's/  */+/g'
| 565248+20480+16384+3297280+147456+8192+20480+32768+217088+4096+36864+28672+\
| 4096+4096+36864+962560+4096+4096+876544+4096+24576+90112+131072
| $ echo $( ( echo 'ibase=16'; \
|             cat /proc/$$/map | awk '{print $1,$2}' | sed -e 's/0x//g' | \
|                 tr '[[:lower:]]' '[[:upper:]]' | \
|                 awk '{print $2"-"$1}') \
|            | bc ) | sed -e 's/  */+/g' | bc
| 6537216
`-----------------------------------------------------------------------

Auto omws einai to VSS tou bash(1), ki oxi to poso mnhmh pairnei to idio
to shell process.  Auto pou leei kai h top(1):

,-----------------------------------------------------------------------
|   PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    CPU COMMAND
| 76648 giorgos       1   8    0  6384K  2260K wait     0:00  0.00% bash
`-----------------------------------------------------------------------

Afairwntas ta /lib/ kai /libexec/ regions apo to virtual memory map tou
process, peftei ais8hta to megethos, alla kai pali den einai idio me to
RSS pou leei h top(1):

,-----------------------------------------------------------------------
| $ echo $( ( echo 'ibase=16'; \
|             cat /proc/$$/map | grep -v /lib/ | grep -v /libexec/ | \
|                 awk '{print $1,$2}' | sed -e 's/0x//g' | \
|                 tr '[[:lower:]]' '[[:upper:]]' | \
|                 awk '{print $2"-"$1}') \
|            | bc ) | sed -e 's/  */+/g' | bc
| 4210688
| $
`-----------------------------------------------------------------------

Me ligo prosoxh mporei kapoios na parathrhsei oti h perioxh mnhmhs pou
einai mapped ws 0xbfbe0000-0xbfc00000 einai to stack tou shell.
Kanontas attach me to gdb(1) sto process kai typwnontas ton $esp
register mporeis na deis oti ontws ekei einai to runtime stack tou.
Ayth omws h perioxh den exei idiaitera megalo megethos:

,-----------------------------------------------------------------------
| $ echo 'ibase=16; BFC00000-BFBE0000' | bc
| 131072
| $
`-----------------------------------------------------------------------

Ara ayth h perioxh einai molis 132 KB.  Opote oi perioxes pou exei
mapped to bash(1), an eksairesei kaneis tis shared libraries einai
4210688-131072 = 4079616, h alliws:

,-----------------------------------------------------------------------
| $ echo 'scale=3; 4079616.0 / (1024*1024)' | bc
| 3.890
`-----------------------------------------------------------------------

Peripou 3.9 MB.

Opote, se ayth thn periptwsh, h mnhmh pou anhkei sto shell einai ligo
megalyterh apo auto pou exei ws 'resident' size.

Giati ginetai auto?  Mporei na ginetai gia pollous logous.

Mporei to leitourgiko na xrhsimopoiei ``demand-paging'', opote kanei men
mmap() to ektelesimo tou bash, alla den to fernei swnei kai kala
automata sth mnhmh thn prwth stigmh pou to trexei kapoios.  Mporei
kapoia merh tou ektelesimou na mh xreiazontai akoma kai na mh xreiastoyn
gia pollh wra.

Mporei to shell na exei fortw8ei olo men, logw tou demand-paging pou
eipa prin, alla na egine swap out, opote to ``resident set'' tou na exei
mikrynei epeidh ena meros tou ektelesimou einai men mapped, alla exei
faei podi apo th fysikh mnhmh.

To RSS pedio pou tupwnei h top(1) (toulaxiston sto FreeBSD pou kserw ti
kanei, afou thn exw peiraksei ki egw merikes fores), to pairnei apo ton
idio ton pyrhna, opote einai ena arketa kalo noumero gia na katalabeis
poso mnhmh _ontws_ xrhsimopoiei ena programma.

Opote, o Miguel exei dikio, alla oxi 100%.

To sigouro einai, oti den einai eykolo na metrhseis AKRIBWS poso mnhmh
xrhsimopoiei ena programma.  Thelei ligo prospatheia, alla toylaxiston
mporeis na empisteyteis _kapws_ to RSS :)