Are the OMSA services running? You will get "Error : Memory object not found" if the OMSA services are not running.
I just tried the command you listed initially on an R620 with OMSA 7.1.0 and am getting the same error: "Error! Specified memory mode not available on this chassis."
However if I use syscfg.exe (part of the Dell Deployment Tookit (DTK)) setting the mem mode works fine: "syscfg --MemOpMode=optimizermode"
Looks like an omconfig issue across the borad or a DIMM issue to me.