NOTE: This disposition is nonprecedential. 

United States Court of Appeals for the Federal Circuit 

2007-3091 

PATRICK N. SWEENEY, 

Petitioner, 

v. 

DEPARTMENT OF HOMELAND SECURITY, 

Respondent. 

Patrick N. Sweeney, of Spring, Texas, pro se. 

Joan Stentiford, Trial Attorney, Commercial Litigation Branch, Civil Division, 

United States Department of Justice, of Washington, DC, for respondent. With her on 

the brief were Peter D. Keisler, Assistant Attorney General, Jeanne E. Davidson, 

Director, and Deborah A. Bynum, Assistant Director. 

Appealed from: United States Merit Systems Protection Board 
NOTE: This disposition is nonprecedential. 

United States Court of Appeals for the Federal Circuit

 

2007-3091 

PATRICK N. SWEENEY, 

 Petitioner, 

v. 

DEPARTMENT OF HOMELAND SECURITY, 

 Respondent. 

___________________________ 

 DECIDED: September 11, 2007 

___________________________ 

Before NEWMAN, Circuit Judge, FRIEDMAN, Senior Circuit Judge, and RADER, Circuit 

Judge. 

RADER, Circuit Judge. 

The Merit Systems Protection Board affirmed the Department of Homeland 

Security’s removal of Mr. Patrick N. Sweeney from his position of Criminal Investigator 

(GS 1811-13). Because the record shows that Mr. Sweeney did not obey a direct order, 

substantial evidence supports the Board's decision and, therefore, this court affirms. 

I 

Mr. Gene D. Lowery, Assistant Special Agent in Charge for National Security, 

Division One, supervised Mr. Sweeney. Before transfer to Mr. Lowery’s division, Mr. 

Sweeney had undergone a fitness-for-duty exam to recover his firearm due to a “public 

safety issue.” Specifically, Mr. Sweeney’s supervisors believed the fitness exam 
necessary because of improper requests for phone records of a prior supervisor, 

harassment of a fellow employee with numerous phone calls “at all hours of the night 

and day” when that employee refused to testify in a lawsuit, attempts to obtain a 

supervisor’s home address, and claims of a "conspiracy" against him by management. 

In the face of a direct order, Mr. Sweeney refused to sign the Authorization for 

the Release of Medical Records in advance of the examination. Mr. Lowery 

recommended Mr. Sweeney's removal for refusing to obey a direct order. The agency 

terminated Mr. Sweeney on March 10, 2006. The Merit Systems Protection Board 

(M.S.P.B.) affirmed the agency’s action. 

II 

This court must affirm any agency action, findings, or conclusions unless they 

are: (1) arbitrary or capricious, an abuse of discretion, or otherwise not in accordance 

with the law; (2) obtained without procedure required by law, rule, or regulation having 

been followed; or (3) unsupported by substantial evidence. 5 U.S.C. § 7703(c) (1996); 

Hayes v. Dep’t of the Navy, 727 F.2d 1535, 1537 (Fed. Cir. 1984). This court examines 

the record for "such relevant evidence as a reasonable mind might accept as adequate 

to support a conclusion." Brewer v. United States Postal Serv., 647 F.2d 1093, 1096 

(Ct. Cl. 1981) (quoting Consol. Edison Co. v. NLRB, 305 U.S. 197, 229 (1938)). 

The Board found that Mr. Sweeney’s misconduct warranted termination Mr. 

Sweeney maintains that the direct order to sign the medical release form violates 45 

C.F.R. § 164.508, the Health Insurance Portability and Accountability Act (HIPAA), and 

5 U.S.C. § 552(a), the Privacy Act. With respect to Mr. Sweeney’s HIPAA claim, HIPAA 

does not provide a private cause of action and clearly authorizes employers to require 

2007-3091 2
disclosure of medical records in connection with a fitness exam. 45 C.F.R. § 164.512 

(b)(1)(v). With respect to the Privacy Act claim, Mr. Sweeney even admitted that the 

Department of Homeland Security had the authority to order him to undergo the fitness 

exam. 

Mr. Sweeney filed a complaint with the Office of Special Counsel alleging, among 

other things, an inappropriate relationship between one of his supervisors (Mr. Stephen 

Coffman) and the daughter of an indicted child pornographer. Mr. Sweeney contends 

that his termination is improper retaliation for “whistleblowing” activities protected by the 

Whistleblower’s Protection Act. 5 U.S.C. § 1211 et seq. The administrative law judge 

(AJ) found no retaliatory motive for the termination. The AJ determined that, even if the 

protected whistleblowing disclosures had not occurred, the agency would have still 

taken the same action. 5 U.S.C. § 1221 (e)(2); 5 C.F.R. § 1209.7(b); Maurano v. 

Department of Justice, 2 F. 3d 1137, 1141 (Fed. Cir. 1993). 

Mr. Sweeney contends that the AJ erred in determining witness credibility. This 

court can overturn an AJ’s decision regarding credibility only if it is “inherently 

improbable or discredited by undisputed evidence or physical fact.” Dittmore-Freimuth 

Corp. v. U.S. 390 F. 2d 664, 685 (Ct. Cl. 1968). Credibility determinations are virtually 

unreviewable. Hambsch v. Dept. of the Treasury, 796 F. 2d 430, 436 (Fed. Cir. 1986). 

Accordingly, this court perceives no basis on this record to disturb the AJ's credibility 

determinations. 

Mr. Sweeney also alleges violations of his Fourteenth Amendment rights of due 

process and equal protection. He asserts that the administrative law judge engaged in 

extra-judicial conduct and exhibited bias. The record provides no basis at all for these 

2007-3091 3
2007-3091 4

claims. Because substantial evidence supports the Board’s decision affirming the 

Department of Homeland Security’s removal of Mr. Sweeny, this court affirms.