Notes will be kept internal only when the field signifies internal or within client details. Notes are visible by all parties involved when they are on shared documents such as invoices and reports.
Notes in the field "Invoice notes" will appear on the invoice if configured this way.