C problem...se megethos domhs

[Aggelos Economopoulos]

On Sun, 25 Jan 2004 05:14:13 +0200
"John Tsiombikas (Nuclear / the Lab)" <nuclear at siggraph.org> wrote:

> Aggelos Economopoulos wrote:
> >>>>Kai genikos anti na kaneis dump ena struct sto disko, einai poly
> >>>>kalyteri idea na organoseis kapos ena fileformat. opote to packing
> >>>ton>structs kai alla tetoia den einai kan issue.
> >>>
> >>>To "na organoseis kapos ena fileformat" stereitai periexomenou.
> >>>Mhpws mporeis na to diatupwseis kalutera?
> >>
> >>tora den katalabes ontos ti eipa i exeis oreksi gia koubenta?
> > 
> > nai, eimai atomo periorismenhs antilhpshs. kane mou th xarh na
> > grafeis mikres protasoules me apla logakia pou den proupo8etoun na
> > skeftomai sto idio context me sena gia na ginoun katanohtes.
> 
> etsi fainete
> 
> >>einai eleino na kaneis memory dump se arxeio, kai profanos mi
> >>efarmosimo se polles periptoseis (px an to structure sou exei
> >>pointers, dynamically allocated members klp), pio anthropini
> >methodos>einai na organoseis ena file structure analoga me ta
> >dedomena sou kai>na exeis mia synartisi pou diabazei apo to arxeio,
> >kai mia na grafei>sto arxeio.
> > 
> > apo pou na arxisw? :)
> > 
> > - uparxoun ena swro oxi-kai toso eleines efarmoges gia tis opoies to
> >   performance metraei polu perissotero apo thn maintainability/
> >   portability tou kwdika, p.x. filesystems, databases.
> 
> to file i/o einai buffered (an xrisimopoieis fwrite/fread klp) eite
> kaneis ena fwrite() olo to struct mazi, eite kaneis 10 mikra fwrite()
> gia kathe member ksexorista, to performance einai idio.

Elpizw na ennoeis /peripou/ idio, giati to performance katafanws *den*
einai idio. An mia struct apoteleitai apo (p.x.) 10 32-bit integers,
katalhgeis na kaneis 10 fwrite() (dhladh ta dekaplasia function calls)
apla gia na doulepsei h stdio wste na ta balei pali se enan buffer kai
na kalesei thn read(). Profanws se ena sunh8hsmeno programma, opou to
i/o den einai to bottleneck, auto den prokeitai na epiferei metrhsimo
slowdown, an omws h berkleydb kai h mysql xrhsimopoiousan apokleistika
thn stdio 8a to eixan kleisei to magazi.

> Kai mi mou les emena gia performance critical apps, mono me tetoia
> asxoloume 

No offence, alla h upar3h an8rwpwn pou den exoun idea gia ton tomea tous
exei papsei na me ekplhsei. Oxi, den ennow oti *esu* den exeis idea
(mporei kai na exeis, mhn kolhsoume se esena twra), ennow oti to "trust
me, I know"(tm) den einai sobaro epixeirhma.

> synithos kai palia ekana diafora kolpa ston kodika gia na paro ligo
> perisotero performance, alla stin teliki ta dirty coding tricks den
> einai ayta pou metrane performance-wise.

Apo edw...

> Prota grafoume ton kodika *katharo* kai meta akolouthontas tis
> symboules enos profiler kanoume optimizations. Kai bebeos ena kako
> data structure tha se xtypisei sto ekatontaplasio sto performance apo
> to na kaneis kati ston kodika pou ek protis opseos mporei na fanei pio
> omorfo alla pio argo. Opote strive for clean code.

... kai ws edw mallon anoi3es to agaphmeno sou programming/softeng book
h apla to apaggeleis apo mnhmhs. Se ka8e periptwsh, auta einai mallon
asxeta me to exm ... "8ema mas".

> > - uparxoun ena swro file formats gia ta opoia einai apolutws xazo
> > (apo
> >   programmatistikh apopsh) na mhn xrhsimopoieis fixed-size structs
> >   gia na ta diabaseis/grapseis eite me xrhsh twn read(2)/write(2)
> >   eite me th xrhsh tou mmap(2), p.x. elf, net protocols. An eisai
> >   tuxeros kai ta format einai kaloftiagmena, den 8a xreiastei pote
> >   na ta dhlwseis san packed.
> > 
> > - an to structure pou 8eleis na grapseis sto disko exei pointers[0],
> >   eisai astoixeiwtos programmer[1] kai auth den einai mia periptwsh
> >   pou mas endiaferei. 
> 
> E oxi me sygxoreis alla i les malakies i exeis ksefygei apo to thema.

could be.

> Ta perisotera data structures xrisimopoioun pointers, ti simenei oti
> den prepei na grafeis ta dedomena ayton ton structures se arxeia
> allios eise astoixeiotos programmatistis?

Ta perissotera data structures einai ftiagmena gia na kratoun in-memory
objects. Mporw na arxisw na sou aradiazw on-disk structures mexri na
kourastw na plhktrologw, alla den blepw to nohma na to kanw otan
amfisbhteis to autapodeikto (endeiktika, apo popular source trees, gia
na mporeis na ta deis kai monos sou:

freebsd (gia thn akribeia ta phra apo to dragonfly pou einai to mono
leitourgiko pou exw proxeiro auth th stigmh):
/usr/include/sys/disklabel.h:struct disklabel
/usr/include/sys/diskmbr.h:struct dos_partition
/usr/src/sys/vfs/ufs/fs.h:struct fs

apo to linux:
fs/xfs/xfs_sb.h:struct xfs_sb
(snipped superblock & inode structs sto fs/*, ta on-disk inode structs
ei8istai na einai embedded sta in memory structs)

(oxi, den einai mono filesystems, koita p.x. ta logs ths berkleydb (an
8umamai kala, bariemai na koita3w))

> PROFANOS de thes na grapseis ta pointers ayta katheayta kai giayto
> tonisa oti prepei na exeis synartiseis pou apothikeyoun ta structures
> sou se kapoio fileformat kai oxi na petas stegna to struct ston disko,
> giati ta pointers theloun special treatment, otan exeis ena struct pou
> exei ena pointer meso tou opoiou kaneis allocate ena chunk mnimis,
> profanos thes na apothikeyseis ta periexomena tou chunk kai oxi to
> pointer (to opoio profanos einai axristo sto arxeio) opote DEN petas
> to struct ston disko. Epidi loipon aytou tou eidous oi domes dedomenon
> einai ypervolika synithismenes, oi periptoseis pou mporeis na kaneis
> memory dump sto disko einai sigoura special case, san geniko kanona
> loipon den kanoume memory dumps ektos apo sygekrimenes periptoseis
> opou 1) mas pairnei 2) to programma einai dirty hack kai den exoume
> diathesi na to organosoume sosta.

h 3) xreiazetai gia logous efficiency
  4) aplopoiei ton kwdika

> 
> > - etc (hdh exoume arketa na suzhthsoume mexri na bare8oume kai oi
> > duo,
> >   eilikrina bariemai na sunexisw)
> 
> Oxi den exo kamia oreksi na to syzitisoume peretero kai idika me ton
> tono syzitisis pou etheses (kai akolouthisa), ego milisa an thes me
> akous an de thes mi soseis, o kathenas einai ypeythinos gia ta
> programmata pou grafei kai gia to styl tou an thes na malakizese go
> on.

Mou epitrepeis na grafw ta programmata mou opws 8elw?!!! Wow!

Sobara twra, an einai na sunexiseis apla gia th xara tou flame den sou
egguwmai oti 8a exw ore3h na diabasw to mail sou. An exeis kati
kainourgio na peis, akouw.

Aggelos