<Style TargetType='{x:Type ListViewItem}'>
<Style.Triggers>
<MultiTrigger>
  <MultiTrigger.Conditions>
    <Condition Property="IsSelected" Value="true" />
    <Condition Property="Selector.IsSelectionActive" Value="true" />
  </MultiTrigger.Conditions>
  <Setter Property="Foreground" Value="Red" />
  <Setter Property="Background" Value="Orange" />
</MultiTrigger>
</Style.Triggers>
</Style>

これでForegroundだけ変わって、Backgroundが変わらないんですが・・・