changed shmall=3500000 and reboot, now it allow me to change max memory to 5000000, but not 6000000.
then change shmall=7500000, then 6000000 allowed for max memory, but not 8630272 (14G)
then I increase shmall = 30000000, then 7000000(2k) = 13.35G max memory allowed.
Thank you so much, Kevin!!! question is: how to calc shmall and how sybase use shared memory and segment?