conjugal rights

Q: Will the husband be sinful if he lives with his wife but does not have any physical relationship with her?