Yes, the SSL certificate slow down the loading speed of your site. This is because the connection has to be encrypted / decrypted on the server and on the client end.