In "Bring Up the Bodies" by Hilary Mantel, Cromwell acknowledges that men like Carew resent him for Anne Boleyn's ascendance, seeing him as the architect of her rise. By orchestrating the downfall of the old marriage, Cromwell allowed Anne to take her place in society. He understands their disdain and realizes they are unlikely to accept him as a friend or ally.
Cromwell's primary desire is not for their acceptance or camaraderie but rather for basic respect. He hopes they won't openly express their contempt for him, especially during shared meals. This illustrates his struggles with the consequences of his actions and the isolation that comes with his political maneuvering.