As well as the ability to preview a theme, you can also update various theme settings all while previewing the changes live. The options that you can change will be dependent on the individual theme.
For the currently active theme, clicking the Customize button allows you to update various settings whilst also previewing those changes. Until you click the Publish button, only the theme preview window is updated, not the theme on your live site.
For all other installed themes that are not currently active, click the Live Preview button to preview the theme, update its settings and activate the theme. Until you click the Activate & Publish button, only the theme preview window is updated, not the theme on your live site.