Layer | Fill | Outline |
---|
Map layers
Theme | Visible | Selectable | Appearance | Zoom Range (now: 0) |
---|
Fill | Stroke |
---|---|
Collaborating Authors
Abstract A parallel adaptive-implicit black-oil reservoir simulator is described which is designed to run on shared-memory machines with the parallelism controlled by OpenMP calls. The basic design decisions leading to this choice are discussed. The changes which were required in the original non-parallel black-oil simulator to achieve parallel execution are discussed, as are the techniques for achieving good parallel speedups for simulator tasks such as building the Jacobian matrix. The simulator uses a parallel implementation of a variable degree incomplete LU (ILU) factorization followed by a parallelized GMRES iteration to solve the linear system of equations associated with the Jacobian matrix. The techniques for achieving this parallelization are discussed, as are techniques for load-balancing some solver sub-tasks. The effect upon the ILU preconditioner of the reservoir decomposition needed to achieve the solver parallelization is discussed and techniques for reducing this effect by reordering within domains are examined. The simulator has been tested on a variety of simulation problems and a number of 32-bit and 64-bit shared memory computers, with the number of processors used varying between 1 and 32. The speedups obtained are presented and analyzed, and the problems entailed in applying the simulator to very large problems (ten million to more than one hundred million active cells) are described. Results are presented for several simulation scenarios as run on several different computers. Introduction The size of reservoir simulation problems run has been traditionally limited by available hardware. The largest problems are currently run on parallel computers. Parallel computers are classified in two categories: shared and distributed memory computers. For distributed memory computers, each CPU has its own memory and communicates with other CPU's through a high speed network. For shared memory computers, each CPU has a shared bus to memory and communicates with other CPU's via a high bandwidth bus, allowing sharing of cache and memory between processors at speeds not possible with distributed memory computers. Early reports of shared memory parallel reservoir simulators included Chien et al. on the Cray XMP. Early reports of distributed memory parallel simulators included Wheeler and Smith on the Hypercube. Over the years, reservoir simulation has moved from being mainly run on supercomputers, to main frames, to mini computers, to workstations/servers. Commonly used workstations/servers for reservoir simulation are based on Intel CPU's and IBM CPU's. Both of these are offered in shared memory configurations. There are advantages and disadvantages of using shared versus distributed memory computers. Optimal performing software needs to be structured differently for shared memory than for distributed memory computers. One advantage of shared memory configurations is that adding an extra CPU is cheaper than buying an additional machine. One disadvantage of distributed memory parallel machines is standard Ethernet networks do not transfer information fast enough to allow good performance. Another advantage, for shared memory computers with adequate bus communication between CPU's, is that arrays need not be over lapped or buffered between domains in order to reduce message passing as has been done for distributed memory simulation implementations. Thus shared memory paradigms use total memory more efficiently. Distributed memory paradigms may use several computers to run problems which would not fit in memory using a single one of those computers, but the ability to continue increasing the problem size is limited as more machines and hence more domains are used. For shared memory, currently OpenMP directives are generally used, while for distributed memory systems, MPI is generally used.
Distinguished Author Series articles are general, descriptive representations that summarize the state of the art in an area of technology by describing recent developments for readers who are not specialists in the topics discussed. Written by individuals recognized to be experts in the area, these articles provide key references to more definitive work and present specific details only to illustrate the technology. Purpose: to inform the general readership of recent advances in various areas of petroleum engineering. Summary Reservoir simulation has moved into a new era with the rapid advancements in parallel-computer hardware and software technologies. In the past decade, the total number of gridblocks used in a typical reservoir study increased from thousands to millions. Mega (million) -cell simulation capability allows accurate representation of reservoirs because of its ability to include detailed information provided by logs, 3D seismic, horizontal wells, and stratigraphic studies. One of the main advantages of megacell simulation is that it minimizes or eliminates upscaling effects, making more-accurate reservoir-performance predictions possible. Furthermore, the speed of parallel computers significantly reduces the manpower-intensive history-matching process. Megacell simulators typically are developed on parallel computers. With the current low-cost parallel computers, a million-cell simulation run covering 20 years of production history takes less than 1 hour. State-of-the-art computer graphics provide an effective means of visualizing large data sets covering million-cell descriptions of a reservoir and its performance. This paper presents an overview of megacell simulation technologies in the oil industry and academia. Introduction The development of parallel computers allowed faster reservoir simulation, placing heavy demand on number crunching and data storage. Parallel computers are classified in two categories: shared- and distributed-memory computers. Shared-memory computers, such as Cray vector supercomputers, are composed of multiple central processing units (CPU's) sharing a common memory. Each CPU of this type of machine is usually very fast and uses vector processing. In contrast, in distributed-memory machines, each CPU has its own memory and communicates with the other nodes (CPU's) through a high-speed network. The number of CPU's in shared-memory machines cannot be increased beyond a specific number (usually 4, 8, or 16), but distributed-memory machines have no such limitations. Distributed-memory machines [sometimes called massively parallel processors (MPP's)] can be made up of hundreds or thousands of CPU's. Each CPU can be very low cost (the cost of a PC); however, the network communication and programming language become major challenges in this type of architecture. Both major and small hardware companies manufacture massively parallel computers. The collection (cluster) of workstations and, more recently, the collection of PC's with a high-speed switch box provide both significant cost savings and attractive performance. Ref. 1 gives a comprehensive description of parallel computers and the associated programming languages, and Abate et al.* discuss the latest efforts and provide an example of a reservoir simulator running on cost-effective PC clusters. Early Oil Industry Efforts As the developments described were taking place in the computer industry, researchers in the oil industry began investigating the benefits of the parallelization in reservoir simulation. The research effort on parallel reservoir simulation during the 1980's is widely discussed in the literature. Shared Memory. Refs. 2 through 4 present early work on parallelization of reservoir simulators for shared-memory/vector computers. The results presented in these works demonstrate the potential benefits of parallel processing. Distributed Memory. Wheeler developed a black-oil simulator on a hypercube, Killough and Bhogeswara presented a compositional simulator on an Intel iPSC/860, and Rutledge et al. constructed a black-oil simulator for the CM-2 and showed that reservoir models with more than 2 million gridblocks could be run on this type of machine with 65,536 processors. The mid-1990's brought further distributed-memory publications. Kaarstad et al. presented a 2D oil/water simulator for a 16,384-processor MasPar MP-2. He showed that a model problem with 1 million gridblocks could be solved in a few minutes. Rame and Delshad parallelized a chemical-flooding code and tested scalability on a variety of systems. Distributed-memory simulators began to move from research to production in 1997 with the presentation of real field applications with massively parallel simulators. Shiralkar et al. presented FALCON, a distributed-memory simulator that uses several programming languages to handle data distribution and interprocessor communication. These languages include high-performance FORTRAN and FORTRAN 90 coupled with message passing [parallel virtual machine or message-passing interface (MPI)]. The simulator was tested on various computing platforms, including TMC CM-5, IBM SP-2, SGI Power Challenge, Cray T3D and T3E, and Origin 2000. Chien et al. presented a distributed-memory simulator based on an existing FORTRAN 77 simulator. They used domain decomposition and MPI message-passing libraries on an IBM SP-2. Their largest reported field was more than 1 million gridblocks running on a 16- or 32-node SP-2 system. Killough et al. presented locally refined grids based on domain decomposition for a distributed-memory simulator. The largest field model featured implicit pressure/explicit saturations (black oil and compositional) for more than 1 million gridblocks and up to 32 SP-2 processors.
- Asia > Middle East > Saudi Arabia (0.71)
- North America > United States > Texas (0.49)
- Energy > Oil & Gas > Upstream (1.00)
- Government > Regional Government > Asia Government > Middle East Government > Saudi Arabia Government (0.31)
An Extensible Architecture for Next Generation Scalable Parallel Reservoir Simulation
DeBaun, D. (ChevronTexaco) | Byer, T. (ChevronTexaco) | Childs, P. (Schlumberger) | Chen, J. (Schlumberger) | Saaf, F. (Schlumberger) | Wells, M. (Schlumberger) | Liu, J. (Schlumberger) | Cao, H. (Schlumberger) | Pianelo, L. (Schlumberger) | Tilakraj, V. (Schlumberger) | Crumpton, P. (Schlumberger) | Walsh, D. (Schlumberger) | Yardumian, H. (ChevronTexaco) | Zorzynski, R. (ChevronTexaco) | Lim, K.-T. (ChevronTexaco) | Schrader, M. (ChevronTexaco) | Zapata, V. (ChevronTexaco) | Nolen, J. (Schlumberger) | Tchelepi, H. (ChevronTexaco)
Abstract We discuss an object-oriented, component-based architecture for a parallel reservoir simulator. The architecture successfully balances the need for extensibility, maintainability, and reuse with the need for efficient computation. Parallelism is hidden from the application developer via a set of abstractions and a unifying framework that supports static and dynamic load balancing on unstructured and structured grids. We explain how the simulator architecture supports black oil and compositional models within a general formulation, and how it achieves parallel scalability for all the major components: property calculation, Jacobian construction, linear solver, well model, input/output etc. Some initial results are discussed for a selection of models ranging from standard benchmarks to large complex field cases. Motivation Current generation reservoir simulators typically have a lifespan of at least 10 years. As a result, many of the simulators in widespread use have their roots in petroleum engineering, software engineering and computing technologies that were originally targeted at structured grids and scalar architectures. Advances in software and hardware technologies, parallel computing, advanced gridding and modern software engineering techniques have created new opportunities. At the same time, evolving workflows involving complex geological models, large full-field simulations, near well studies, production and facilities optimization, and stochastic modeling have placed new demands on reservoir simulation technology. Although object-oriented design has been the main staple of software development for quite some time, its application to scientific programming has been hindered somewhat by the perceived inefficiency of run-time polymorphism. However, in recent years the emergence of compilers that are in full compliance with the C++ standard has made use of generic programming techniques commercially practical. Generic programming offers the benefits of encapsulation and code reuse without the overhead of virtual function calls. Object-oriented and generic programming techniques using C++ have been shown to be useful in the development of scientific and numerical applications. Large-scale object-oriented frameworks for scientific computing have been developed in many contexts, and parallelism has been addressed for example in the POOMA project. Object-oriented architectures for flow simulations have been developed in other industries and these techniques have been applied to reservoir simulators by several practitioners within academia and also in industry . Introduction In order to take advantage of the trends in reservoir simulation and computing technology described earlier, we have invested in a next generation simulator. The key features of this simulator, referred to as Intersect (or IX) are:parallel scalability the ability to handle large and/or complex reservoirs unified treatment of structured and unstructured grids multi-purpose implementation adaptable software architecture. coupled surface networks In this context, complexity can arise from modeling of structure, advanced wells, heterogeneity, compositional fluid behavior etc. Underpinning our simulator is extensive research in collaboration with industry and academia on: architectural frameworks; formulation of nonlinear equations in the reservoir and wells; linear solvers; parallel communication models; compositional phase behavior; object oriented design and framework development; unified well model; load balancing; and longer term initiatives such as multi-scale modeling. We chose an approach that takes advantage of the object-oriented and generic programming capabilities of C++ to provide maximum reuse and extensibility without sacrificing computational efficiency.
Abstract The multiscale, multimesh flow simulators have been designed for the sole purpose of running very large, heterogeneous reservoir, flow problems on massively parallel computers. This paper shows the flow simulation results and the corresponding CPU times. The multiscale flow simulator is written in Fortran 90/95 with OpenMP directives and compiled on high-performance SMP computers. The simulations were performed for several highly heterogeneous, channelized, reservoir cases with realistic rock-fluid interaction (viscous, capillarity, gravity, and compressibility) to evaluate the efficacy of the multiscale, multimesh simulation in parallel computing. It is shown that the aforementioned multiscale technique reduces computing time by several orders of magnitude, while maintaining the same accuracy as the conventional fine scale simulation. Introduction In simulation of displacement processes in large heterogeneous reservoirs, the computing time is both time-consuming and expensive. Therefore there has been a tendency to upscale fine-grid models to reduce the required CPU time. The problem with upscaling is that it often creates inaccuracy in mathematical results (e.g. large numerical dispersion). Upscaling also cannot capture the architecture of the flow channels effectively. Thus, the channeling effects are suppressed. Finally the upscaling algorithm usually does not have a solid physical foundation. For instance, permeability upscaling has been handled through a logical flow averaging technique; however, upscaling of relative permeability curves has not been very well developed. As a consequence, to minimize the upscaling issues, we have resorted to a multimesh, multiscale computing methodology to preserve the reservoir flow and transport characteristics at the very fine-level, while we reduced the inherent computing time by several orders of magnitude. The multiscale computation was reported by several authors previously.7โ17 We also presented an extension of the multiscale method for both single- and dual-porosity reservoirs in a previous meeting.5โ6 Since then, we have been able to improve our computing methodology, which is the subject of this paper. The multiscale, multimesh simulator was compiled for a 64-bit, SGI-ALTIX with 256 1.5 GHz Itanium2 CPUs. However, for the purpose of this study, we limit our usage with a maximum of 32-CPUs. Computing Methodology We solve the steady-state pressure equation on the global fine-grid mesh, to obtain the flux distribution at the coarse-grid boundaries. These flux distributions are used as the weighting function for the local pressure update instead of the transmissibility weighting used in our previous. We also use the above steady-state fine-grid flux distribution at the boundaries of the coarse-grid to calculate the effective permeability tensor of the coarse-grid. This upscaling approach is different than the classical flow-based permeability upscaling which is based on constant pressure at the boundaries. The latter approach is also equivalent to having a fixed pressure gradient across the coarse-grid domain. The computation sequence:Obtain global fine-scale steady-state pressure solution to calculate the fine-scale flux-weights at the boundaries of each coarse-scale nodes. This information will be used to calculate the fine-scale fluxes within each each coarse-scale nodes. For computational efficiency, for very large grid systems, we use block Jacobi iteration algorithm. For parallel processing purposes, block Jacobi iteration can be done by a red-black ordering scheme. Obtain global unsteady-state coarse-scale pressure solution at large time-step, ?t1, to calculate the coarse-scale fluxes. Calculate the the unsteady-state fine-scale fluxes at the coarse grid boundaries using the coarse-scale fluxes and weighted by Step a. Calculate the fine-scale pressures and internal interface fine-scale boundaries within each coarse-scale gridblocks using the boundary conditions obtained in Step c. Calculate fine-grid saturations using smaller time-steps, ?t2, constrained by the CFL criterion for the IMPES or sequential approach.
A Systematic Approach To Evaluate Hydrocarbons In Thinly Bedded Reservoirs
Passey, Q.R. (ExxonMobil Upstream Research Co) | Dahlberg, K.E. (PetroGeeks LLC) | Sullivan, K.B. (McKinsey & Co.) | Yin, H. (ExxonMobil Upstream Research Co) | Xiao, Y.H. (ExxonMobil Exploration Co.) | Guzman-Garcia, A.G. (ExxonMobil Exploration Co.) | Brackett, R.A. (McKinsey & Co.)
ABSTRACT Many oil and gas reservoirs contain significant volumes of recoverable hydrocarbons in intervals where the average bed thickness is below the resolution of conventional logging tools. The presence of thin beds increases the uncertainty in all approaches to hydrocarbon pore thickness estimates. Standard cutoff methods cannot be used to determine net pay reliably for thinly bedded intervals. The presence of thin shales has a disproportionate effect on resistivity log measurements, reducing their sensitivity to the presence of hydrocarbons. Conventional log-analysis methods, including classical shaly sand methods, typically underestimate hydrocarbon volumes in such thinly bedded intervals, often by 30% or more. A further complication induced by thin bedding is macroscopic anisotropy, which causes resistivity logs to measure different average values depending on the relative angle between the borehole and bedding planes. In ExxonMobil, a two-pronged approach has been developed to improve the evaluation of in-place hydrocarbons in thinly bedded reservoirs:For intervals where the average bed thickness is greater than about one foot, a highresolution measurement, such as an image log, is used to define individual bed boundaries and construct a detailed earth model. Using the high-resolution earth model, detailed 1-D and 2-D forward models are generated for resistivity logging tools and simplified 1-D models are produced for porosity and lithology logs. Modeled responses are compared to measured log responses, and the earth model is adjusted until a match between the modeled and measured logs is obtained. For reservoir intervals where the average bed thickness is below one foot, a unique lowresolution approach is used without attempting to resolve individual beds. Here, a statistical earth model for the entire thinly bedded interval is formulated. This model comprises two or more bed types and assumptions about the petrophysical properties and relative frequencies of each type. Mathematical models then predict the averaged log responses over the interval, accounting for the effects of thin bedding and macroscopic anisotropy. A Monte Carlo inversion method is used to find the ranges of bed properties and frequencies that yield matches between the predicted and measured log averages, thus, resulting in an improved estimate of in-place hydrocarbons, including an estimate of the uncertainty. This systematic, two-pronged approach has been used in reservoir evaluations worldwide and has resulted in improved, and often significantly increased, estimates of in-place hydrocarbons in thinly bedded reservoir intervals. The result of either interpretation can be compared to that derived from NMR analysis when these logs have been acquired in the well of interest. Many oil and gas reservoirs contain significant volumes of recoverable hydrocarbons in intervals where the average bed thickness is below the resolution of conventional logging tools. The presence of thin beds increases the uncertainty in all approaches to hydrocarbon pore thickness estimates. Standard cutoff methods cannot be used to determine net pay reliably for thinly bedded intervals. The presence of thin shales has a disproportionate effect on resistivity log measurements, reducing their sensitivity to the presence of hydrocarbons. Conventional log-analysis methods, including classical shaly sand methods, typically underestimate hydrocarbon volumes in such thinly bedded intervals, often by 30% or more. A further complication induced by thin bedding is macroscopic anisotropy, which causes resistivity logs to measure different average values depending on the relative angle between the borehole and bedding planes. In ExxonMobil, a two-pronged approach has been developed to improve the evaluation of in-place hydrocarbons in thinly bedded reservoirs:For intervals where the average bed thickness is greater
- Geology > Sedimentary Geology > Depositional Environment > Transitional Environment (1.00)
- Geology > Geological Subdiscipline > Stratigraphy (1.00)
- Geology > Rock Type > Sedimentary Rock > Clastic Rock > Mudrock > Shale (0.47)
- Geology > Rock Type > Sedimentary Rock > Clastic Rock > Sandstone (0.32)
- Africa > Nigeria > Gulf of Guinea > Niger Delta > Niger Delta Basin > OML 133 > Erha Field (0.89)
- Africa > Nigeria > Gulf of Guinea > Niger Delta > Niger Delta Basin > OML 133 > Bosi Field (0.89)