Using the side buttons on the handhelds does not turn off the sound of the device, it only turns down the ringer as the majority of these devices were designed to be used as phones.
To completely mute the device, the following steps must be taken.
1. Go to the Sound and Notifications settings (Start > Settings > Sound & Notifications)
2. Un-check the Events and Programs boxes, click OK.
And that will do it! Your device should now be muted.