While building an attachment.php template for a project I opted to customize the loop to properly display an image within a figure tag and the caption within figcaption to aid accessibility.

Additionally if the file is not an image it will provide a button that can be styled along with a mime type.

The following code would have to be placed within the loop.

<?php 
if ( wp_attachment_is_image() ) {

	// Set the image's ID to an array so it can be reused later
	$image_id = get_the_ID();
	$caption = wp_get_attachment_caption( $image_id );

	echo '<figure>';

		// Displays the large version of the image if available
		echo wp_get_attachment_image( $image_id, 'large' );

		// if no caption is set or the value is null do nothing and don't display the empty tags
		if (!empty($caption) ) {
			echo '<figcaption>' . $caption . '</figcaption>';
		}

	echo '</figure>';

} else {

	// If the attachment is not an image display a button and mime type.
	$attachment_id = get_the_ID();

	$attachment_url = wp_get_attachment_url( $attachment_id ); 

	echo '<button type="button"><a href="' . $attachment_url . '" target="_blank">' . get_the_title( $attachment_id ) . '</a></button> ';

	echo '<span class="mime-type">' . get_post_mime_type( $attachment_id ) . '</span>';

}
?>