Currently it looks like this (gamma exaggerated for illustration purposes):

If I convert it as-is, the size increases because of the JPEG artifacts.
Example:
Original JPEG: ipsum.jpg, 10KB
Converted to PNG: ipsum.png, 14KB
What's the best way to reduce the noise but without distorting the text? (I rather keep some noise than remove some detail from the text)