[roman]

En el formulario modal intenta capturar el mensaje WM_SYSCOMMAND para desde ahí minimizar la aplicación:

Código Delphi [-]
interface

type
  TModalForm = class(TForm)
    { ... }
  private
    procedure WMSysCommand(var Message: TWMSysCommand); message WM_SYSCOMMAND;
  public
    { Public declarations }
  end;

implementation

procedure TModalForm.WMSysCommand(var Message: TWMSysCommand);
begin
  if Message.CmdType and $FFF0 = SC_MINIMIZE then
    Application.Minimize
  else
    inherited;
end;

LineComment Saludos