I went through the same steps and ended up with loading ssm Params at the startup of the lambda exactly as you mentioned in this story.

Do you have any suggestions for caching here? Is there any serverless plugins that would work seamlessly with ssm?

Thanks